<geist>
zid: interesting. i think the overflow thing is indeed because qemu is doing a queing thing
<geist>
inside the driver the ethernet subsystem has a 'e1000_can_rx' call that apparently causes it to essentially never trigger a actual RXO irq because the upper layers of qemu just holds off and queues packets (or doesn't read from the tun0, etc)
<geist>
so that's fie, but it means as soon as i start adding packets back to the tail it instantly pushes the head forward
<geist>
which is also fine, but it doesn't seem to trigger any additional irqs
<geist>
so then it just deadlocks. you end up with RDH = RDT and no irqs to push the engine forward
<geist>
the e1000 qemu driver doesn't have this problem, it seems to kick tiself forward, though it has a similar problem of never triggering a RXO
<geist>
it's like there's a missing edge in their logic that says if packets are already queued up and you bump the tail forward, receive it but forget to set irq status
<geist>
orrrr,it has something to do with the fact that it precisely triggers an IRQ at the time i'm adding stuff to the RDT and interrupts are disabled in that window in my code and so the irq gets lost somehow due to MSI shenanigans...
<geist>
like maybe i have it set up as level when it should be edge or something, and the latching effect is lost
<zid>
so just e1000e things?
<zid>
good debugging though
<geist>
e1000 seems to continue, though i dont get a RXO because of the qemu upper layer holding off
<geist>
but i'm also not using MSIs there
<geist>
yah so e1000e qemu driver seems to be trying to set the irq as expected, but it's not making it to the cpu. curious!
sdfgsdfg has quit [Quit: ZzzZ]
freakazoid343 has quit [Ping timeout: 240 seconds]
xing_song has quit [Read error: Connection reset by peer]
xing_song has joined #osdev
wille has quit [Ping timeout: 256 seconds]
ZombieChicken has quit [Quit: WeeChat 3.3]
ZombieChicken has joined #osdev
scoobydoo has quit [Read error: Connection timed out]
scoobydoo has joined #osdev
wille has joined #osdev
bradd has quit [Ping timeout: 240 seconds]
bradd has joined #osdev
pretty_dumm_guy has quit [Quit: WeeChat 3.4]
wille has quit [Ping timeout: 268 seconds]
xing_song has quit [Read error: Connection reset by peer]
xing_song has joined #osdev
wille has joined #osdev
freakazoid12345 has joined #osdev
bradd has quit [Ping timeout: 240 seconds]
bradd has joined #osdev
bradd has quit [Ping timeout: 268 seconds]
gog has quit [Ping timeout: 268 seconds]
<vin>
Has anyone dabbled with RDMA before? Can you register two memory regions between peers simultaneously in one sided RDMA?
<vin>
this is on infiniband
<zid>
I'm too brokeass to have touched infiniband
<moon-child>
no but it looks cool, I hadn't heard of that
<vin>
yup I get 54 Gb/s on a single port but have to rethink memory management for apps.
xing_song has quit [Read error: Connection reset by peer]
xing_song has joined #osdev
Maka_Albarn has joined #osdev
radens has joined #osdev
<Maka_Albarn>
I'm getting a very weird jump in memory when trying to move a pointer. it keeps jumping from 0x10e004 to 0x115ff8 when it should be going from 0x10e004 to 0x111ffc.
<bslsk05>
github.com: qemu/e1000e_core.c at master · qemu/qemu · GitHub
<bslsk05>
github.com: qemu/e1000e_core.c at master · qemu/qemu · GitHub
<geist>
basically if it decides to postpone the irq because of the ITR timer (holds off irqs until a time period ahs fired) then in some deep path the timer fires which then tries to queue all postponed irqs
<kazinsal>
oh ho, good find
<geist>
but, it comes back in via this function, which has already marked in `core->msi_causes_pending` that it has sent it
<geist>
and thus skips it
<geist>
my little cheesy hack whichs eems to work is to save a copy of core->msi_causes_pending at the top of the function, and restore it if it decides to postpone the irq
<geist>
and that seems to work fine
<geist>
somewhat annoyingly, e1000e does not let you disable timers via setting ITR to 0 like you can on e1000
<geist>
so you have to always set at least some time to delay irqs
<geist>
msix works differently with a different set of timers, so it's probably why no one has noticed the problem until now
<geist>
because linux/etc probably uses MSIX all the time
<geist>
you have to be precisely using MSI and not legacy and not MSIX, which is probalby very not tested
<kazinsal>
I'm still waiting for VirtualBox to support MSI on e1000
<kazinsal>
(they won't)
<geist>
honeslty i didn't need to dive into this this deeply but this is how you Really learn how stuff works
<geist>
if your driver works the first time you only get one level into the guts of things but as a result of this i really had to try to grok how the whole ITR stuff works
<geist>
and it's pretty nice
xing_song has quit [Read error: Connection reset by peer]
<geist>
so the race condition seems to be that within a timeout window you have to queue a IRQ, then have the guest start handling it, then queue another packet such that the second one doesn't generate a new MSI. I can see why that's hard to reproduce: the usual timeout window is just a handful of microseconds, so the guest has to really get on top of it. LK is fast enough to catch the window i guess
ZombieChicken has quit [Remote host closed the connection]
<jason1234>
I have managed to compile the LK kernel
* sahibatko
got rid of all external deps for the bootloader, yay!
<jason1234>
however, how to create a ISO , qcow2 and/or img file for boot.
<jason1234>
I would like too qcow to try on our VPS OPENBSD ircnow
radens has quit [Quit: Connection closed for inactivity]
<geist>
can create a grub image and load it that way i suppose
<geist>
I rarely do, but i know that works fine
<geist>
so whatever the normal mechanism for creating a disk image with grub install and then add a menu item to boot the lk.elf or lk.bin would work fine
<geist>
normally i do most work on arm or riscv or other arches which dont have the grub thing
Reinhilde is now known as Ellenor
<kazinsal>
for rapid kernel testing I boot over PXE
<geist>
but i've been piddling with x86 more, so i really should probably fill that particular hole, indeed
<geist>
kazinsal: what is the precise sequence you use there? loading grub over PXE to load your thing or do you do a PE binary thing for UEFI?
<kazinsal>
grub over PXE, then the grub.cfg points the root at the (tftp) device
<geist>
yah probably worth doing. for some local hackery to boot on old devices i have either a bootsector and floppy image generation tool or i load grub on a usb stick and toss lk.bin on it
<kazinsal>
there's a grub-mknetdir script that lets you generate a full set of everything GRUB needs to boot over PXE
<geist>
but i dont have an automated way to create that except i did once and i just sort of keep it around and use mtool to update the fs
<geist>
yah, jason1234 is probably on openbsd so i have no idea if any of that works there
<kazinsal>
maaaaaaybe? might need some hackin'
<geist>
right
<geist>
as far as iso images, probably? but honestly ive never tried
<geist>
usually on x86 i either load from a grub disk if on real hardware or i use -kernel on qemu
<geist>
huh. odd. rebooted back into windows 10 for the first time in a week or so and now it's telling me it's ready for a win11 update
<geist>
but.... last i checked it was not eligable, mostly because i have secure boot disabled
<geist>
now i'm worried they figured out how to set it bavk
<kazinsal>
I think they dropped some of the weirder requirements
<geist>
i'm guessing what happens is if i started to try to update it'd force me to enable secure boot
<geist>
i think it think my machine is secure boot capable so it's cool with starting the upgrade
<kazinsal>
apparently there's a Rufus update that lets you burn a Windows 11 ISO to a USB with a custom option to disable TPM and secure boot requirements
<graphitemaster>
Heh, finding out from a contact that the chip shortage has lees to do with high demand / low supply, and more to do with the backlog of shipping containers.
<graphitemaster>
Apparently one of the ports off of LA has about 4 years worth of containers unprocessed.
<geist>
yah seattle has a similar problem i think
<kazinsal>
yeah vancouver's got the same kind of issue and also weird coastal currents resulting in things like Barge Chilling Park
<kazinsal>
(there's a runaway barge beached on the shore of English Bay and the Vancouver Park Board put up a sign to commemorate it)
<graphitemaster>
The backlog is apparently a logistical nightmare, even if they hired on 2000 new employees to process 24/7 at the port in long beach, it would still take about 2 years to process all the current stuff assuming no new ships arrived. They have no clue what to even do, the bubble that exists cannot be caught up to.
<graphitemaster>
That's frightening.
<ZombieChicken>
Sounds like a great time to start shipping via Zepplin.
<sahibatko>
or finally get to building the underwater bullet train railway
<kazinsal>
announcing the AeroTrain!
<sahibatko>
Crap, still have not figured out how to enable some -haccel with qemu on windows host. Enabled Hyper-V feature, disabled it, it never works with long mode on guest.
<sahibatko>
in many ways, life is much simpler in the nix world.
<kazinsal>
iirc you need to install Intel HAXM
<kazinsal>
and turn on hyper-v
<kazinsal>
then qemu should be able to take -accel hax as an argument
<sahibatko>
that sounds promissing
<ZombieChicken>
Is it theoretically possible to write one driver that could run on potentially any microkernel? I'm sure there are a few gotchas in there, but the question came to mind
<Affliction>
theoretically, sure!
<Affliction>
With enough #ifdefs anything's possible!
<moon-child>
ZombieChicken: obviously not, given a sufficiently broad definition of 'microkernel'
<moon-child>
so the real questino is: how broad is _your_ definition of 'microkernel'?
<ZombieChicken>
moon-child: I guess just pondering the idea of a standardized driver interface, or at least compatability layers so that a driver from kernel A could run under kernel B
<moon-child>
oh
<moon-child>
udi?
<Affliction>
Well, you could build an abstraction layer between the core of your driver and the kernel.
<Affliction>
That abstraction layer might be implemented in the kernel, or in the driver
<bslsk05>
wiki.osdev.org: Uniform Driver Interface - OSDev Wiki
<Affliction>
In much the same way as yuo'd have an abstraction layer to make an app portable between OSes
YuutaW has quit [Quit: WeeChat 3.3]
<ZombieChicken>
moon-child: Figures. Yet another idea someone else came up with
<ZombieChicken>
ty
<moon-child>
well. It's dead
<Affliction>
oh, while osdev is back up, let's see if it's possible to grab an xml dump
<moon-child>
(that said, its banes are likely to also hamper any of your efforts to craft an equivalent)
<kingoffrance>
ZombieChicken, also oskit (also dead) and xfree86/xorg used to say same architecture/cpu drivers could run across oses
<ZombieChicken>
banes?
<moon-child>
banes. Killers
<kingoffrance>
wolfsbane. zombiechicken bane
<kingoffrance>
batman bane
<ZombieChicken>
ok
<kingoffrance>
actually it might be bain
<moon-child>
it is not
<kingoffrance>
:)
<Affliction>
XML dump acquired - opted out of all page history though.
<Affliction>
If the site falls over again, I'll re-learn how to mediawiki and host it myself
YuutaW has joined #osdev
diamondbond has joined #osdev
<Belxjander>
ZombieChicken: I'm looking at Windows and Linux driver code for making them work on another microkernel host but I also have to contend with the target host needing multiple architecture support as well
<Belxjander>
ZombieChicken: there is partial driver layer stuff for things like that too "ndiswrapper" beiing an example explicitly for Windows Network Device drivers to be usable by Linux
<Belxjander>
in dgos-master... where in the hell is "PAGESIZE" actually defined !?!?
freakazoid12345 has quit [Ping timeout: 240 seconds]
gog has joined #osdev
nyah has joined #osdev
sdfgsdfg has joined #osdev
wolfshappen has quit [Quit: later]
tryte has joined #osdev
tryte has left #osdev [#osdev]
sdfgsdfg has quit [Quit: ZzzZ]
ahalaney has joined #osdev
dennis95 has joined #osdev
pretty_dumm_guy has quit [Ping timeout: 256 seconds]
pretty_dumm_guy has joined #osdev
immibis has quit [Read error: Connection reset by peer]
pretty_dumm_guy has quit [Client Quit]
blockhead has quit []
dude12312414 has joined #osdev
wolfshappen has joined #osdev
shikhin has quit [Quit: Quittin'.]
shikhin has joined #osdev
scoobydoo has quit [Read error: Connection timed out]
scoobydoo has joined #osdev
someonejv has quit [Ping timeout: 256 seconds]
wolfshappen has quit [Quit: later]
wolfshappen has joined #osdev
wolfshappen has quit [Ping timeout: 240 seconds]
wolfshappen has joined #osdev
scoobydoo has quit [Read error: Connection timed out]
scoobydoo has joined #osdev
xing_song has quit [Read error: Connection reset by peer]
xing_song has joined #osdev
dude12312414 has quit [Remote host closed the connection]
dude12312414 has joined #osdev
bauen1 has quit [Ping timeout: 260 seconds]
ElectronApps has quit [Remote host closed the connection]
bauen1 has joined #osdev
not_not has joined #osdev
<not_not>
hi
<not_not>
where should i start design phase on drawing board on OS? sound and gui first? im having a really hard time thinking outside the box due to programming tunnel vision
warlock has quit [Remote host closed the connection]
<not_not>
but the kernel and bootloader should be designed first right, thinking x86 64 bit, tried writing a bootloader but got stuck in protected mode just before i could start printing error messages on screen in 32 bit mode
pretty_dumm_guy has joined #osdev
<not_not>
but i also want to think about the future
<j`ey>
processes and filesystems and drivers
<not_not>
what new architectures are comming up in the future
<not_not>
j`ey: ok thank you
<not_not>
multi core ofc
bauen1 has quit [Ping timeout: 240 seconds]
<not_not>
and ofc NO BUGS
<not_not>
on multicore does each core have its own cache and the stack is often located in cache?
<j`ey>
they have caches per-core and shared caches
<not_not>
ok, how can you optimize caches?
<not_not>
is it possible to have an entire cache dedicated to iptr return addresses so there is no physical way of data buffer overflows to overwrite iptr?
<not_not>
you also discuss programming languages here?
<not_not>
wait forums says there is a user called Owen, he wouldnt happen to be from san fransisco?
<kingoffrance>
there is a separate unrelated #proglangdesign but that is more designing a lang perhaps
<j`ey>
im pretty sure you dont have that level of control over the cache
<not_not>
bah
<not_not>
ait kingoffrance ill join that one too cuz i have a great idea for a programming language
xenos1984 has quit [Quit: Leaving.]
novasharper has quit [Quit: You have been kicked for being idle]
xing_song has quit [Read error: Connection reset by peer]
xing_song has joined #osdev
ajoberstar has joined #osdev
zaquest has quit [Remote host closed the connection]
ravan_ has quit [Read error: Connection reset by peer]
<bslsk05>
hackaday.com: Building MS-DOS From Scratch Like It’s 1983 | Hackaday
<sonny>
for those designs with a shared address space, do you get less memory for programs than when using virtual addresses?
mahmutov has joined #osdev
freakazoid12345 has joined #osdev
dude12312414 has quit [Quit: THE RAM IS TOO DAMN HIGH]
dennis95 has quit [Quit: Leaving]
scoobydoo has quit [Read error: Connection timed out]
scoobydoo has joined #osdev
sonny has quit [Ping timeout: 252 seconds]
ravan has joined #osdev
Belxjander has quit [Ping timeout: 260 seconds]
ravan has quit [Quit: Leaving]
sonny has joined #osdev
ajoberstar has quit [Ping timeout: 268 seconds]
YuutaW has quit [Ping timeout: 268 seconds]
YuutaW has joined #osdev
<geist>
sonny: right
<geist>
though you can use virtual addressing and shared address space and at least carve up a single large virtual address spae
<sonny>
ok, then that's a huge downside
<sonny>
ohhhh
<geist>
ie, 2^47 bits of a single address space even if the computer only has say 2^36 or something physical memory
<geist>
in the 80s or so that was a bigger win, because you'd have a full 4GB virtual address space but probably only 4 or 8MB ram at best
<jimbzy>
I remember those days.
<geist>
so i guess another way of putting it is virtual memory doesn't imply multiple address spaces, but you probalby need some sort of hardware assist to implement MAS
<sonny>
so when doing design in this area, it's either how to address memory or how to keep memory seperate per process?
<jimbzy>
We used to sit around in our log cabins looking out our DOS4GW.
<sonny>
I see, I kinda got that impression from my book
<sonny>
jimbzy what does gw mean?
<sonny>
4 gigabit word?
<sonny>
nah lol
<geist>
dos 4G W
<geist>
I think the W was maybe the vendor, i forgot there name
<sonny>
also when you start to look at virtual memory, that determines how security is handled in your whole os I think
<geist>
sonny: note that virtually all modern mmu capable OSes do a fairly traditional multiple address space scheme
<geist>
where each process gets their own address space
<jimbzy>
GeDaMo, cheers.
<geist>
and as such hardware is optimized out the wazoo for that, so the cost of switching aspaces isn't that big of a deal
dude12312414 has joined #osdev
<sonny>
I see
<geist>
(was just preemptively pointing out it's not that expensive, since it's usually the next things folks talk about)
<geist>
i've been down the conversation here tons of times, though not as much lately, about how SAS could be so much faster etc etc
<geist>
and i think it's a forest for the trees thing, where folks like to optimize for one thing (mmu context switching) at the expense of basically everything else
<sonny>
I guess it would be faster 40 years ago, but times have changed :D
<geist>
right
<geist>
the amount of cache, prefetching, etc on modern cores really mitigates the cost of MMU context switching
<sonny>
what you want to get right now is the page table size
<geist>
plus all modern hardware has some form of per process tagged TLBs
<sonny>
that or have hardware with a large TLB
<geist>
yah, though that kinda works against you if you have to dump it on every context switch
<geist>
on the original 386 for example it had 16 TLB entries, period
<sonny>
you don't, that's what the process tags are for?
<geist>
so it was lame, but then the cost of dumping it was not that high because it was only 16 entries in the first place
<sonny>
lol
<geist>
right. process tags + huge number of TLBs mitigates it
<sonny>
16 entries
<sonny>
hmm, such a crucial aspect but all the problems are sovled :P
<geist>
original 386 didn't even have a INVLPG instruction
<geist>
so the only way to invalidate a single TLB entry was to dump the whole thing (reloading CR3)
<geist>
that was added in 486
<geist>
i discovered this when trying to load my kernel on a real 386 :)
<sonny>
oops lol
<sonny>
hmm, I must say it's always a block for me whenever x32 comes up, reading about this DOS extender thing
<geist>
huh there was literally a coyote just off my porch right now trying to catch a squirrel that i had to shoo off
<geist>
this one is pretty bold
<kazinsal>
meanwhile it took another 30 years for us to get an instruction to INVLPG on other cores
<geist>
kazinsal: yah reminds me i really should look into addding PCID support. the first gen of it didn't seem that useful but on modern stuff (zen and things post skylake) it seems finally generically useful where it's a win/win
<geist>
also for that matter the new zen 3 tlb shootdown stuff looks choice.
<sonny>
s/x32/x86/
<kazinsal>
yeah, invlpgb on zen 3 does broadcast TLB shootdown for free
<kazinsal>
well, "free"
<kazinsal>
no IPI needed but I'm guessing it probably has limitations in big clustered systems
<geist>
it's almost to the letter the same mechanism arm64 has
<geist>
thus further proving that a lot of the ryzen design came out of their aborted K12 ARM core design
<geist>
thus there's just a whole backchannel bus required to do those shootdowns, and ARM seems to be all in on it as well, even for massive systems. it's probably hard to get that right, but fundamentally not much different than cache shootdowns and whatnot
<geist>
may even use the same underlying cache coherency machinery
<kazinsal>
neat. arm64's on my eventual to-do list. I never actually learned 32-bit ARM so it'll be a nice clean slate for me
<geist>
kinda curious now that you mention it. i have highly technical docs for the thunderX1 cpus, which describe the whole cpu-cpu bus and everything, so would be an interesting window into at least one implementation
<geist>
since you will of course never get intel or amd docs for cpu to cpu transfers anymore
<kazinsal>
totally. it's hard enough to find concrete numbers on those
<geist>
kazinsal: yah the only thing different on the ryzen one is they have a single instruction that can flush a range of addresses, whereas the ARM one is one instruction per page
<geist>
but, in both cases the key is the flush instruction is asynchronous: you fire it and then you have to run another instruction to wait for it to complete
<geist>
so it's nice because you can flush pages as you go while doing some larger transaction and then sync at the end
<kazinsal>
that's handy
<kazinsal>
I spent a solid hour the other night trying to figure out what the actual bus width of intel's current gen inter-socket interconnect is
<kazinsal>
they publish the GT/s but not the bus width so there's no concrete definition of "this is how fast a UPI lane is"
CaCode has joined #osdev
rustyy has quit [Quit: leaving]
rustyy has joined #osdev
sortie has quit [Quit: Leaving]
mahmutov has quit [Ping timeout: 268 seconds]
sortie has joined #osdev
GeDaMo has quit [Remote host closed the connection]
<catern>
hmm
<catern>
it just occurred to me that... so with readiness-based interfaces, there's the classic thundering herd problems
<catern>
where you wake up everyone who might want to perform an operation, and it wastes CPU because only one of them is actually allowed to do it
<catern>
Unix has this a lot, or another example is just condvars
<catern>
the obvious solution which is implemented in some places is to only wake up *one* waiter, and they just get to do it
<catern>
what occurred to me is that that's basically a completion-based API - you're basically just giving the resource to the one waiter who woke up
<catern>
(and also it doesn't work super well if the only waiter you woke up could then crash, without forwarding the notification to anyone else...)
<catern>
has anyone written anything about readiness-based-but-you-only-wake-up-one-waiter interfaces? i've obviously seen lots of readiness vs completion arguments but surprisingly little about readiness-but-you-only-wake-up-one-waiter
<j`ey>
I guess you do something when a process crashes
CaCode has quit [Quit: Leaving]
ajoberstar has joined #osdev
sonny has quit [Ping timeout: 256 seconds]
Dreg has quit [Read error: Connection reset by peer]
ahalaney has quit [Remote host closed the connection]
ThinkT510 has quit [Quit: WeeChat 3.3]
CaCode has joined #osdev
ThinkT510 has joined #osdev
sdfgsdfg has joined #osdev
ZombieChicken has quit [Quit: WeeChat 3.3]
CaCode_ has joined #osdev
CaCode has quit [Ping timeout: 256 seconds]
sonny has joined #osdev
gog` has joined #osdev
gog has quit [Ping timeout: 268 seconds]
gog` is now known as gog
UsbSick has joined #osdev
xing_song has quit [Read error: Connection reset by peer]
xing_song has joined #osdev
ajoberstar has quit [Quit: ERC 5.4.1 (IRC client for GNU Emacs 29.0.50)]