<netbsduser>
it has a feature called 'DAX' (not Dax J, that's the DJ) that lets you map the host's page cache for a file such that it appears as physical memory to the guest
<netbsduser>
all good and well for old-fashioned read() and write(), but a trickier thing for mmap() ironically. linux uses it to optimise read()/write()
<netbsduser>
but if there's an mmap() request on a file in a virtio-fs, and read()/write() are being doing through dax, i wonder if they take any measure to avoid incoherence with respect to the guest page cache (which i presume still backs mmap), or do they wrangle a way to let the dax windows serve the mmap() requests?
sinvet has quit [Remote host closed the connection]
sinvet has joined #osdev
Matt|home has quit [Quit: Leaving]
airplanemodes has joined #osdev
<netbsduser>
looks like they have special treatment of it so they can serve mmap() requsts properly. it's also a feature which cries out for huge pages support, something which will be difficult for me because i rely a lot on pages always being nicely PGSIZE-sized
ol__ has joined #osdev
sinvet has quit [Read error: Connection reset by peer]
fedorafan has quit [Ping timeout: 252 seconds]
<mrvn>
Unless you have a shared read-only mapping I don't see how huge pages would apply.
<mrvn>
I also wonder why mapping the hosts page cache is a problem for mmap?
fedorafan has joined #osdev
<netbsduser>
it's simply my preference not to allocate lots of `struct vm_page`s if i can avoid it
<netbsduser>
and i would need to allocate them in this instance because everything else depends on them being around
<mrvn>
how do you mmap without those structs?
<netbsduser>
you can't
<mrvn>
which kind of invalidates your preference
<netbsduser>
quite untrue, that's where large pages would come in
<netbsduser>
my preference is to avoid allocating lots of them, i would consider 512-fold reduction in that number to not be 'lots' anymore
<mrvn>
a) that would still require a struct and b) you do not want to mark a huge page dirty
<mrvn>
You need one struct per page? You don't have something to map a page range?
<netbsduser>
like most kernels all pages of main memory are described by a struct
<netbsduser>
you may map things which aren't main memory without needing such a struct (a framebuffer, for example)
gog has quit [Ping timeout: 260 seconds]
<mrvn>
well, you can't have your cake and eat it too
<mrvn>
It also sounds to me you would only need a single alloction of "struct vm_page[]" on mmap. Just one array, not one alloc per page.
<netbsduser>
oh, they aren't allocated on mmap()
<netbsduser>
they are allocated at startup to describe all main memory. they track things like which queue they're on for page replacement, reference count, what logical object they belong to (an anonymous page, page cache of a vnode, etc), and suchlike
<mrvn>
consider it memory hotplugging.
<mrvn>
Someone mmaps 1GB via host-fs is like someone plugged in an extra 1GB of memory.
nyah has quit [Quit: leaving]
<netbsduser>
my issue is that needing to allocate 16 mibs worth of vm_pages (which have to sit in actual main memory) per gib of dax-mapped virtio-fs file is unpleasant to me. i want to reduce that overhead if i can, so that how much you can map with DAX is less limited by available main memory
<mrvn>
a valid concern. Can you make a more compact vm_page struct for that case?
<mrvn>
Do you need more than a present, accessed, dirty bit per page?
<netbsduser>
it's not a bad idea. some things vm_page stores are not particularly relevant to the case of dax
<mrvn>
page replacement I think doesn't apply if it's host memory. reference count would be per page range. logical object again per page range.
<mrvn>
Does DAX handle pageing in blocks on access when they aren't in the hosts cache?
<netbsduser>
yeah, it breaks the logic of page replacement a bit because (apart from the overhead of the vm_page structs) you are not paying anything for them. my reading of the virtio spec suggests it does do page-in on host on fault
<mrvn>
Sounds like it really just mmaps the file into the qemu process and you get a new block of memory that's backed by the file.
<mrvn>
You might not even have to do anything about present/accessed/dirty bits because the host will do that already.
<netbsduser>
one thing i need to figure out is what (if any) hard limit there is on how much you can map with dax; if there is limited space (other than the actual guest physical address space, which is not a limiting factor for me yet) then i need some form of replacement mechanism, but that would be a dax-specific mechanism
<mrvn>
netbsduser: not sure that makes sense. A replacement mechanism would make this a whole lot more difficult and then you do need a per page struct again. Try mapping a few terrabyte and then call it good enough.
<netbsduser>
mrvn: well, given the inexpensiveness of mapping with DAX, and assuming it provides a relatively large area for mapping, a simple fifo of (fixed-size chunks of) dax mappings could suffice
<netbsduser>
when one has to fall of the end, wipe it from every address space it's mapped in
<netbsduser>
i have a vague notion now as well of how i can address one area where i depend on vm_page structs; namely in how i wire down buffers for I/O. those call the fault-handler with a special code to tell it to wire the pages and return a pointer to the struct
<mrvn>
first check if it's limited oher than the address space size.
<netbsduser>
for DAX mappings the structs can be allocated lazily
<netbsduser>
they can also go away when no one is wiring them anymore
airplanemodes has quit [Ping timeout: 252 seconds]
dude12312414 has joined #osdev
FreeFull has quit [Remote host closed the connection]
dude12312414 has quit [Quit: THE RAM IS TOO DAMN HIGH]
<kazinsal>
geist: sorry for the ping brother but a friend of mine is wondering what the situation is like down there with regards to how easy it is to score a case or three of vanilla coke. shit is basically worth its weight in gold up north of the 49th
wootehfoot has joined #osdev
wootehfoot has quit [Max SendQ exceeded]
wootehfoot has joined #osdev
sortie has quit [Quit: Leaving]
sortie has joined #osdev
gog has joined #osdev
<gog>
mew
<Ermine>
hi gog, may I pet you
<gog>
yes
* Ermine
pets gog
Gooberpatrol_66 has joined #osdev
Gooberpatrol66 has quit [Ping timeout: 246 seconds]
* kazinsal
gives gog scritchies
mctpyt has quit [Ping timeout: 265 seconds]
nyah has joined #osdev
bauen1 has quit [Ping timeout: 248 seconds]
* gog
prrr
<zid>
gogpurs
fedorafan has joined #osdev
<zid>
for riding cats
MrBonkers has quit [Quit: ZNC 1.8.2+deb2build5 - https://znc.in]
jjuran has quit [Quit: Killing Colloquy first, before it kills me…]
<mrvn>
This item cannot be shipped to your selected delivery location. Please choose a different delivery location.
<mrvn>
What's in there that they override free speach?
craigo has joined #osdev
<jjuran>
Of the four reviews displayed, the one-star and two-star reviews seem to have legitmate complaints, whereas the two five-star reviews offer no substance.
baine has joined #osdev
<ilovethinking>
The author assumes some knowledge that a person unfamiliar with the subject would likely not know. Not only that, but the author introduces concepts, then does not explain them very well, if at all. Yet, there are problems/review questions on them. There are even questions on content covered (in the chapter appendix) after those questions are presented. The author does a good job of making it seem
<ilovethinking>
as though the book is well written until you actually read it.
<ilovethinking>
i think the book is made to assume people who read it know what they are reading
<mrvn>
This looks like a text book for an university course. Pay them 200k and visit the lectures and he will explain.
<ilovethinking>
i'm reading it and it's fine for me
<ilovethinking>
i guess it isn't that good for undergrads
<zid>
That is infact how reviews work
<ilovethinking>
?
<mrvn>
There is a Global Edition and an International Edition as well. I wonder what's different in them.
<jjuran>
The Home Edition is cheaper, but go for the Ultimate Edition if you can afford it
<mrvn>
International is cheapest and prices go up to 8 times that.
<ilovethinking>
idk i found the book in my uni
<ilovethinking>
no one uses them because like
<ilovethinking>
yeah
<ilovethinking>
makes sense
<mrvn>
ilovethinking: "An introduction to operating systems" probably shouldn't assume much knowledge, at least about OSes.
<ilovethinking>
fair point
<mrvn>
I would even expect them to explain something like a linked list because "providing readers with a solid understanding of the key structures and mechanisms of operating systems"
<mrvn>
Anyway, that book is 6 years old. Other than as introduction I would think it will be quite outdated.
<mrvn>
"the practical decisions affecting design, performance and security" have changed a lot in the last 6 years. :)
ilovethinking has quit [Ping timeout: 246 seconds]
<netbsduser>
much of it is timeless and perpetually relevant
ilovethinking has joined #osdev
gog has joined #osdev
<ilovethinking>
mrvn: could you recommend any books? i mean i already know how oses work and im developing one but i just want to make my foundation and knowledge stronger
* ilovethinking
welocmes gog
<gog>
hi
<netbsduser>
i think you will find books on extant OSes useful
<gog>
goðan daginn til öllar stýrikerfiðsforritarana
<ilovethinking>
netbsduser: extant?
<gog>
bækur fyrir stýrikerfi sem til
<netbsduser>
ilovethinking: really existing
<netbsduser>
(like how erich honecker described the political system of east germany as being)
<gog>
lesaðu til að læra
<mrvn>
ilovethinking: I would say go to your library and read the books they have. I wouldn't spend $100-$200 on a book that may or may not be any good and you are probably only going to read once.
<ilovethinking>
i have a kindle so i don't mind paying a few dallas
<mrvn>
ilovethinking: I only saw hardcopy version of the book.
<ilovethinking>
the one i sent yeah
<ilovethinking>
but i mean i don't mind paying a for a book you could recommend
<CalimeroTeknik>
a completely random idea: is there an OS where the audio driver is a stream (e.g. unix character) device that stops consuming input when the jack is unplugged?
<CalimeroTeknik>
effectively pausing audio
<mrvn>
For a foundation I can recommend Andrew S. Tannenbaum. But that recommendation is from decades ago.
<ilovethinking>
thanks
<mrvn>
CalimeroTeknik: that would assume the hardware has a way to check if a jack is plugged in
<ilovethinking>
ofc im not looking for something very beginnerish but i just want to understand certain things better, especially userspace/memory/storage related
<ilovethinking>
but i think the one we have at our library is fine
<ilovethinking>
it covers virtual memory and a bit of userspace
<mrvn>
ilovethinking: focus on the idea and not the details and probably any book will get you started.
<ilovethinking>
yeah well i have like vmm/pmm/timer/etc done but yeah you get my point
<ilovethinking>
ill just read that one
<mrvn>
ilovethinking: a good knowledge of algorithms and data structures helps a lot too
<ilovethinking>
yup i discovered that haha
<ilovethinking>
(the hard way)
<CalimeroTeknik>
mrvn, fair enough, then I can rephrase to "if we know that the jack is plugged out"
<mrvn>
CalimeroTeknik: No idea if any OS converts that hardware signal into blocking. Doesn't seem like a good way. The device will block if you write to it too fast or too much at a time. So when it blocks you don't quite know what's going on. Maybe a signal would be better.
<netbsduser>
ilovethinking: that's very quick
<netbsduser>
after 9 months my VMM is still in much flux, there is always more to do
<CalimeroTeknik>
good point mrvn. The first idea that comes to mind is, "either write() succeeds immediately or fails immediately" (if you ask for O_NONBLOCK or something)
<CalimeroTeknik>
and then the program needs some way to wait on the device accepting more data
<netbsduser>
the last big thing was swapping, and the next will probably be swapping kernel data structures: tmpfs node descriptions should be; the contents are already anonymous VM objects
<netbsduser>
the support structures for anonymous mappings also need to be swappable, otherwise virtual memory is limited by physical, which is ridiculous
<mrvn>
CalimeroTeknik: succeeds immediately would normaly be EOF
<mrvn>
netbsduser: is it? Given the difference in speed for ram and disk and ram prices does swapping make sense anymore?
<gog>
CalimeroTeknik: that's better resolved by an application that pauses playback when an audio device change is detected on its output
<gog>
unless you mean input then it would pause recording
<gog>
but idk how to resolve the fact that it might get garbage for a few milliseconds before the pause happens
<gog>
maybe just know the latency beforehand and rewind the recording that much
<CalimeroTeknik>
both, and I was hoping some poll kernel API of the sort that I don't know of, already existed
<netbsduser>
mrvn: it seems to me the gap has dramatically narrowed
slidercrank has quit [Ping timeout: 255 seconds]
<netbsduser>
these NVMe SSDs operate at fantastic speeds
<CalimeroTeknik>
right, the garbage makes sense, but I suppose this is a driver problem
wootehfoot has joined #osdev
<gog>
laws of physics problem too
<CalimeroTeknik>
admittedly
<mrvn>
netbsduser: you only have really high speed SSDs on really high speed clusters so the gap is still there I feel
<gog>
idk about any applicaiton that does that already, but i imagine it'd be possible to do that with pipewire
<gog>
i don't know its api though
<netbsduser>
the case for swap is helpfully also supportable by the existence of significantly slower access to secondary than main memory
<netbsduser>
under those conditions it's criminal to leave data in memory which won't be accessed for a while while it could be put to better use caching other data on disk
bauen1 has joined #osdev
<mrvn>
netbsduser: my experience in HPC tells me that you tailor your code to really not swap for best results. And you want fast IO to stream data through your algorithms instead of mapping data sets way larger than ram.
<ilovethinking>
what can it mean if all of my code started PFing randomly, like literally when loading the idt and setting idt[cnt] = descriptor it triplefaults
<mrvn>
netbsduser: overall the reason I don't like swapping (or more generally mmap) is the practically impossibility to deal with errors.
<ilovethinking>
when everything used to work fine and i made no changes to the idt code
<mrvn>
netbsduser: and if you take that out of your memory design it gets so so so much simpler.
<mrvn>
ilovethinking: your double fault handler is broken. Fix that now.
<ilovethinking>
mrvn: i can;t load a handler without an idt
<ilovethinking>
and the idt triplefaults when loading
<ilovethinking>
it's the stupidest thing
<mrvn>
ilovethinking: qemu or bochs can show you all three faults. decode them by hand.
<ilovethinking>
it's a pf + df
<ilovethinking>
that's a triplefault rihgt
<ilovethinking>
right
<mrvn>
where is the #pf? Did you mess up the address? Is that page mapped wrong? debug it.
<ilovethinking>
i don't map shit
<ilovethinking>
nothing happends
<ilovethinking>
happens*
<ilovethinking>
it's pagefaults almost on kernel entry
<ilovethinking>
_start calls load_idt() and load_idt() pagefaults
<mrvn>
19:41 < mrvn> where is the #pf? Did you mess up the address? Is that page mapped wrong? debug it.
<netbsduser>
mrvn: it's a fair objection, and certaunly for e.g. a database there is a reason they prefer to not use mmap
<ilovethinking>
mrvn: how can a page be mapped wrong/address messed up when i don't map anything
<mrvn>
netbsduser: I think so too. Leaves more work for applications that might want to swap but my experience is that when stuff swaps performance usually tanks so quick that it's not worth it (for me at least).
<netbsduser>
i also managed to dig up some information on virtiofs with dax (in practice this whole thing seems to be tailormade for Linux so they barely even document it in the virtio spec); the dax region appears to be fixed-size and set at the time of launching qemu
<mrvn>
Apps that can swap intelligently work well. But the kernel will swap out your mouse cursor and totally screw you.
<mrvn>
ilovethinking: a page fault is litterally the thing you get for not mapping things. So why are you surprised.
<mrvn>
?
<ilovethinking>
man i don't fucking know
<ilovethinking>
limine is supposed to map the stack for me
<ilovethinking>
but it doesn't
<ilovethinking>
fuck limine
<ilovethinking>
all my homies hate limine
<ilovethinking>
(add the all my homies hate meme here)
<mrvn>
is it? How big is the stack?
<ilovethinking>
can it be because i ask it to gimme bss
<ilovethinking>
no wait but i removed it
<ilovethinking>
hmm
<mrvn>
no. "rsp is set to point to a stack, in bootloader-reclaimable memory, which is at least 64KiB (65536 bytes) in size, or the size specified in the Stack Size Request (see below). An invalid return address of 0 is pushed to the stack before jumping to the kernel."
<pitust>
limine does map the stack[1]
<mrvn>
So what is in RSP?
<mrvn>
Oh, fun. This is going to totally screw you if you don't consider it: "If 5-level paging is requested and available, then 5-level paging is enabled"
<ilovethinking>
im so tired of limine for some reason
<ilovethinking>
it provides you shit and then you have to switch all of it at some point
<mrvn>
but I assume you aren't requesting that.
<ilovethinking>
and it become pointless
<mrvn>
ilovethinking: it still saves you that intermediate step
<ilovethinking>
i tried setting up grub-mkrescue on mac but i couldn't do it
<ilovethinking>
because well
<ilovethinking>
mac
<mrvn>
ilovethinking: we are all still waiting for you to give up limine like you said you would
<ilovethinking>
IM TRYINGGG
<ilovethinking>
what bootloader do i use
<mrvn>
what bootloader are you using for load your real system now?
<ilovethinking>
wdym "load real system"
<pitust>
given he has a mac im gonna say iBoot
<mrvn>
the OS where e.g. your irc runs
ol__ has quit [Remote host closed the connection]
<ilovethinking>
ah
<pitust>
limine is really quite good though
<ilovethinking>
i don;t fucking know do you think mac let's me know that
<pitust>
mrvn: If 5-level paging is **requested**
<ilovethinking>
let me call up timmy
<pitust>
and if you requested you can see if it enabled it
<pitust>
you don't actually need grub-mkrescue to set up grub though
<pitust>
you kinda need it for a nice isohybrid
<pitust>
but you dont even need it to make just hard disks for bios or uefi
<gog>
ilovethinking: which part of that code are you wondering about for the idt
<ilovethinking>
which one
<gog>
the idt
<ilovethinking>
oh the trap one
<gog>
yeah
<gog>
it's a macro to generate ISR stubs
<ilovethinking>
i don;t understand why
<ilovethinking>
why not set up seperate gates
<gog>
you do set up separate gates
<gog>
the stub will call a higher-level handler
<pitust>
you have to unless you want to create some horribleness
<gog>
yes
<ilovethinking>
fair
<ilovethinking>
thanks
<gog>
i've tried to do the horrible way and it's not worth it
<gog>
the stub way has many advantages
<pitust>
i have compiled the stubs at runtime once for the meme
<gog>
well meme'd
<pitust>
there are other cursed things i know of you can do around the interrupt handling
<gog>
i love cursed things in code
<pitust>
like tss-free usermode with the power of syscall
<gog>
oh that's very cursed tell me more
<pitust>
i haven't done it but i've heard of the concept
<gog>
oh
<gog>
how would i tknow what stack to switch to on interrupt
<pitust>
~~not yet anyway~~
<pitust>
if you don't change cpl while taking the interrupt, you don't need a tss
<mrvn>
pitust: how is that going to work? A stack overflow is an instant tripple fault.
<pitust>
so if your interrupt handler is in cpl=3, then that handler can use syscall to get to kernelmode
<gog>
ohhhh
<ilovethinking>
mrvn you still didn't tell me what bl to use
<ilovethinking>
left me hanging like that
<pitust>
limine is a pretty good bootloader
<mrvn>
ilovethinking: whatever you are already using
<ilovethinking>
wtf are you talking about
<ilovethinking>
im using limine and i don;t want limine
<pitust>
it (a) mostly doesn't matter unless you are rolling your own horribleness and (b) switching is likely not worth the effort
<mrvn>
ilovethinking: I'm not going to tell you what to use. make up your own mind please.
<gog>
i have a theoretical problem with handling interrupts in PL3
<mrvn>
pitust: you need a TSS for exceptions.
<pitust>
PL3?
<gog>
what happens if another interrupt comes in before you syscall
<gog>
CPL=3
<mrvn>
gog: you leave interrupts off
<pitust>
gog: the power of trap gates
<gog>
you need CPL=0 to clear RFLAGS.IF
<gog>
i think
<pitust>
if=0 cpl=3 is a valid rflags combination
<mrvn>
gog: i believe the gate has bits for IF
<gog>
hmm
<mrvn>
gog: without that there would always be a race condition between the IRQ happening and disabling irqs.
<gog>
that's true
<gog>
does the user code also have access to the IDT
<mrvn>
On the other hand how do you return from an IRQ?
<pitust>
iret is still valid in usermode
<mrvn>
but that then allows user mode to disable IRQs by setting IF=0 on the stack.
<mrvn>
clearly that doesn't work
<mrvn>
How are IRQs forced back on on iret?
<pitust>
i'm pretty sure its like any other instruction that sets rflags
<mrvn>
meaning?
<gog>
ok i need to look at the manual for awhile because i keep having more questions
<pitust>
mrvn: either #GP or the bits get ignored
<pitust>
it would be silly if you could get access to io ports with popfq
<mrvn>
IF CPL ≤ IOPL
<mrvn>
THEN EFLAGS(IF) := tempEFLAGS; FI;
<pitust>
oh so the bit won't set
<mrvn>
What's the IOPL for an IRQ?
<pitust>
iopl doesn't change
<mrvn>
CPL is 3, so if IOPL is 3 then you can change EFLAGS
<mrvn>
If IOPL is less than on iret IRQs won't be enabled and you are screwed.
<mrvn>
s/than/then/
<pitust>
oh thats an issue
<mrvn>
either way you have a problem.
<pitust>
so there is a race condition between sysret and iretq
<pitust>
unless sysret forms an interrupt shadow which i doubt
<pitust>
no ofc it doesn't
<pitust>
actually who says the iret has to happen in userspace?
<pitust>
enter the kernel with syscall, exit with iretq
<mrvn>
You could allow nested IRQs. syscall/sysret has it's own IF flag handling so that works fine.
<mrvn>
syscall would be CPL=0
<mrvn>
You still have the problem that user code can screw up ESP and then you tripple fault.
<pitust>
yeah
<mrvn>
That's actually the only thing you normaly use the TSS for in long mode: Set valid stacks for CPL=0 and stack/double faults. Everything else in the TSS you leave blank.
bitterlollipop has joined #osdev
Arthuria has quit [Ping timeout: 260 seconds]
baine has quit [Ping timeout: 248 seconds]
baine has joined #osdev
bitterlollipop has quit [Quit: system sleep - ZZZzzz...]
Matt|home has joined #osdev
<ilovethinking>
is bootboot a good bootloader
<ilovethinking>
idk wtf to use lol
slidercrank has joined #osdev
<ilovethinking>
is there another way to make a grub iso without grub-mkrescue
bitterlollipop has joined #osdev
bitterlollipop has quit [Client Quit]
<gog>
if you do mkisofs with the right directory structure you can load grub with efi
<bslsk05>
gcc.gnu.org: Other Builtins (Using the GNU Compiler Collection (GCC))
<gog>
it's here just not much is said about it
<tacco>
It is not?
<gog>
floating-point math is very implementation and platform-dependent
baine has quit [Ping timeout: 255 seconds]
<gog>
fma
<tacco>
yeah but no word about __builtin_fma
<klange>
"All these functions have corresponding versions prefixed with __builtin_."
<gog>
The ISO C99 functions ... are handled as built-in functions except in strict ISO C90 mode (-ansi or -std=c90).
<gog>
fmaf is listed there
baine has joined #osdev
<tacco>
oh the sentence about "prefixed" does apply to the preceding paragraph as well? I guess that was my mistake. Thank you!
<klange>
Be careful with the math __builtin_s, they are all defined to _maybe_ inline some platform instruction to do the thing... and failing that will call the libm version.
<klange>
So if you are using them to implement the libm version, you need to check that the ones you are using are actually doing the thing for your platform.
<tacco>
uhhhh thanks for the warning
danilogondolfo has quit [Remote host closed the connection]
Turn_Left has joined #osdev
Left_Turn has quit [Ping timeout: 252 seconds]
heat has joined #osdev
<heat>
hell
<klange>
is other people
<heat>
netbsduser, re: DAX, DAX has been a thing for the last 5-6 years on Linux for non volatile mem devices
<heat>
pretty much designed so things like optane et al can just have a large 2TB persistent mmio window which gets easily randomly accessed
<heat>
DAX basically skips having a page cache in the middle, so you do your read/write/mmap/etc on DAX memory itself (if you look at docs, funnily enough it says DAX currently only supports filesystems with block size = PAGE_SIZE)
<heat>
and yeah, this stuff usually does not have a struct page
<heat>
a 2TiB storage device with a 64 byte struct page would result in ~34GiB of struct pages
<netbsduser>
that figures, if it was a smaller block size and someone mmap()s a file which begins on a non-page aligned address/is a non-page-aligned length you've given over to the mmap()'er pages they have no business touching
<netbsduser>
given over to them blocks they've no business touching, rather
<heat>
which is either super wasteful or absolutely impossible (like on a desktop system where you could concievably have a similar storage dev but little RAM)
<netbsduser>
heat: what i find curious is how it interacts with virtio-fs which is intended only for mapping smaller windows, and in any case is absolutely limited in how big the region you can map into is
<heat>
why?
<netbsduser>
presumably it doesn't just go "sorry, no space left, you can't mmap() till someone else munmaps()/sorry, we'll have to do mmap() the slow way again"
<heat>
why would mmap matter for space?
<netbsduser>
because there is only a limited amount of space available for you to ask virtio-fs to map windows into
<heat>
this DAX looks weirdly non-functional
<heat>
pretty much just a fast path thing?
<puck>
looks like it to me too
mctpyt has joined #osdev
<puck>
like, you can't just not implement FUSE_READ/FUSE_WRITE
<heat>
i was expecting virtio-fs to just map all the files in physical memory
<heat>
... although I guess that does come with its own set of problems given file size changes
<puck>
virtiofs is just FUSE over virtio lol
<heat>
yep
<netbsduser>
oh yeah, the spec is a joke
bgs has quit [Remote host closed the connection]
<netbsduser>
"please refer to the linux kernel and libfuse"
<netbsduser>
"the API is undocumented but you'll figure it out"
<puck>
i understand the reasoning behind doing this; i wish it'd specify a bit more but tbf fuse is a stable enough uapi
<heat>
damn right
<heat>
fuck all the non-linux people
<netbsduser>
you know, for all i know, virtiofs dax might actually not at all be intended to support mmap() in the guest
mctpyt has quit [Ping timeout: 246 seconds]
<puck>
indeed
<puck>
heat: eh, most BSDs i care about have fuse compat layers
<puck>
and it's probably better than the alternative of NIHing a fuse-alike
<heat>
i'm disappointed openbsd has a fuse
<heat>
smh
<puck>
wait openbsd has it?
<puck>
huh.
<heat>
yes puc.
<heat>
puck*****
<netbsduser>
trying to look at linux's virtiofs implementation to see whether they use DAX to help with mmap also
<puck>
virtio-gpu is getting drm kernel api support also (which, to be fair, isn't very big, and every other *nix-alike basically reuses the kernel abi or something very similar)
<heat>
even the BSDs you don't care about have fuse compat layers
<puck>
netbsduser: dax is more generic than just virtiofs tho
<heat>
yes but in this case probably not
<heat>
puck, virtio-gpu's 3D stuff is also terribly documented
<netbsduser>
i figure they can either A) support mmap of virtiofs directly with dax, but need some kind of specialised dax-window-replacement-algorithm; or b) not support it, which introduces another two possibilities
<heat>
"either use DRM and mesa or use them for docs" is a shit policy
<puck>
heat: to be fair virgl is a mess in general
<heat>
they have also fumbled the whole thing
<heat>
it's 2023 and we still don't have an upstream vulkan solution?
<netbsduser>
if they don't support backing mmap()'d regions of files with dax windows then either a) they suffer an incoherent page cache for the mmap()'d stuff of virtiofs files, or b) they try to pick pages from their own page cache before going to to DAX to read/write
<heat>
also, the DRM kernel API is IMO huge
<heat>
(and DRM itself)
<puck>
heat: i've considered adding cross-domain to virgl and it's quite a chaos
<puck>
i need to sit down for a good day with a debugger and just trace the virgl code i think
<heat>
mesa has some tracing stuff now that may help
<puck>
this is the virglrenderer side, not mesa
<heat>
ah
<puck>
honestly i'm surprised that virtio-gpu didn't start with drm passthru from the start, but instead doing the GLX thing
<heat>
because drm passthrough sucks?
<puck>
which, speaking of, also, what is the GLX protocol even. it's horribly documented
<puck>
heat: kinda, but it's /fast/
<heat>
I would've done something like expose vulkan as the lowest common denominator of low level crap
<heat>
definitely too early for GL-on-vulkan
<heat>
although I don't know if this was too early for vulkan
heat has quit [Remote host closed the connection]
<puck>
virglrenderer is ancient.
heat has joined #osdev
<puck>
virglrenderer is ancient.
<heat>
virglrenderer is ancient.
<puck>
like. 2014
<puck>
heat: hey, i assumed you missed that message the first time
<heat>
i did but I also checked the logs :D
<heat>
>2014
<heat>
>ancient
<heat>
technically almost true but man does that upset me
<puck>
heat: relatively, for virtio stuff? i'd say that's Pretty Old
<puck>
heat: anyways they have hit the issue that now they need to map gpu buffers into guest memory
<puck>
hence the everything getting more complex
<puck>
..wait, was the issue iommus
<puck>
Huh.
<heat>
one day/week/month I'll need to sit down and port drm
<heat>
and honestly, fuck that
<heat>
otoh, good idea
<puck>
hrmm
<puck>
did fuchsia port drm?
<heat>
no
<heat>
they just stole some good bits of mesa
<heat>
I assume stuff related to shader compiling?
<heat>
that sounds a helluva lot like a PITA
<netbsduser>
managarm os did their own DRM
<puck>
right, it kinda makes less sense to do the exact drm api
<netbsduser>
i think they might have virtio-gpu too but i might be confabulating
<puck>
virtio-gpu isn't hard for 2d accel only
<heat>
yeah but fuchsia's API is not DRM
<puck>
hrmm
<heat>
they actually wrote new drivers and a new API for graphics
<puck>
now i wonder if wiindows has an equiv to the "map buffer", "create gpu buffer", "submit command buffer to gpu" set of calls
<heat>
those silly people, with their hard work
<puck>
windows*
<puck>
heat: i wonder if the magma intel command buffers are compatible with linux i915 drm
<netbsduser>
ffs i can't even google anything to do with 'dax' without getting the wrong results
<puck>
heat: which is a bit more specific a question :p
<netbsduser>
'dax reclaim' (a mechanism that apparently exists) just returns Dax J's Reclaim Our Lost Honour
<xvanc>
rip
<gog>
jadzia dax
<netbsduser>
dax j famously got into trouble in morocco, or maybe tunisia, i forgot
<xvanc>
osdev is no match for the power of Dax J
<netbsduser>
he mixed the islamic call to prayer into his set
<heat>
DAX PERFORMANCE-INDEX
<netbsduser>
github as usual tells lies
<bnchs>
hi os endeveours
<netbsduser>
"We couldn’t find any code matching 'reclaim' in torvalds/linux"
<bnchs>
netbsduser: github's search feature is dogshit
<bnchs>
sometimes it can't find something, but when you search the same thing again in the same repo, it SUDDENLY finds something
levitating has quit [Ping timeout: 255 seconds]
<netbsduser>
bnchs: i nowadays refresh a second after the first search beacuse usually i get double as much 2nd time around
<netbsduser>
and often if i get to e.g. page 2, then when i click to go to the next page, which it claims exists at the time, it then says "sorry, no more results"
<bnchs>
anyway how's the os devils
levitating has joined #osdev
<netbsduser>
bnchs: going well
xvanc has quit [Quit: leaving]
<bnchs>
good
xvanc has joined #osdev
tacco has quit [Remote host closed the connection]
gxt__ has quit [Remote host closed the connection]