<heat>
bhyve not having a qemu integration is a little silly
<heat>
can it not support qemu at all or did just no one put in the work?
darkstarx has quit [Ping timeout: 246 seconds]
craigo has quit [Quit: Leaving]
coolcoder613 has quit [Ping timeout: 252 seconds]
darkstardev13 has quit [Quit: Leaving]
netbsduser has joined #osdev
stolen has quit [Quit: Connection closed for inactivity]
bauen1 has quit [Ping timeout: 252 seconds]
memset has quit [Remote host closed the connection]
memset has joined #osdev
vinleod has joined #osdev
adder has joined #osdev
X-Scale has joined #osdev
vdamewood has quit [Ping timeout: 252 seconds]
X-Scale has quit [Ping timeout: 256 seconds]
the_oz_ has quit [Remote host closed the connection]
the_oz_ has joined #osdev
<immibis>
sgx was always broken due to the bugs
<nikolar>
sgx?
<goliath>
nikolar, intel secure enclave
<nikolar>
ah
memset has quit [Remote host closed the connection]
memset has joined #osdev
<goliath>
In the news recently, because someone managed to extract keys. IIRC 5+ years ago someone showed that you could use speculative execution to make SGX trigger page faults, based on data you weren't supposed to see.
<immibis>
AFAIK you can single-step the encrypted code. It would make sense to check what changed after each instruction - even if you can only see which addresses changed and not what the new value is
bauen1 has joined #osdev
edr has joined #osdev
<netbsduser>
something seems to have magically fixed my kernel under kvm on aarch64
<netbsduser>
previously i was getting bizarre and inexplicable synchronous exceptions with ESR=0x2000000 when executing demand-paged userland code
<netbsduser>
despite an overabundance of barriers and icache invalidation
<netbsduser>
at first i thought i fixed something incidentally but can't even reproduce on old commits
<netbsduser>
so the only remaining explanations are the edk2 build i used, the qemu version or something else updated on the pi, the pi firmware that i updated, the date, fate, or god's mysterious will
<nikolar>
just don't worry about it :)
m3a has quit [Ping timeout: 248 seconds]
m3a has joined #osdev
hwpplayer1 has joined #osdev
steelswords has quit [Read error: Connection reset by peer]
Arthuria has joined #osdev
steelswords has joined #osdev
Bonstra has quit [Ping timeout: 248 seconds]
levitating has joined #osdev
<Ermine>
TIL they still host kde 1 and 2 sources
<nikolar>
weren't those closed source
<Ermine>
no
levitating has quit [Read error: Connection reset by peer]
<Ermine>
qt was (partly) non-free
<nikolar>
ah just qt, not kde
goliath has quit [Quit: SIGSEGV]
<heat>
netbsduser, is there a non-crap POSIX aio implementation?
<heat>
(and fwiw i've heard it has significant issues cuz it needs signals)
<Ermine>
Can you have a good implementation of crappy api?
<netbsduser>
heat: a lot do it with a kernel thread pool instead of userland threads, some might do it as actual async I/O without threads
<heat>
i mean, if you look at it objectively, no. if you look at it relative to other impls, yes
<netbsduser>
any OS worth its salt offers something other than signals too
<vinleod>
The least bad is the best by default.
<heat>
>some might do it as actual async I/O without threads
<heat>
this is actually Generally Not Possible soooo
<nikolar>
is it not
<heat>
no
<netbsduser>
aix has completion ports, solaris might offer them for it (surely it does?), freebsd adds a kqueue filter for it
<heat>
nikolar, consider this: i need to read data but to do that i need to read the btree and act upon it
<netbsduser>
heat: i would accept only that it's Hard to do it for filesystems
<heat>
how do i do that async?
<netbsduser>
and that it's dubious whether the work necessary to allow FS I/O to be asynchronous is worth it
<netbsduser>
for raw disk or partition reads though, triflingly trivial
<nikolar>
^
<heat>
isn't that, uh, what windows nt fans say is super good and great
<heat>
raw disk or partition reads don't actally matter in the grand scheme of userspace programs
<netbsduser>
i checked the only public fs driver of windows (FastFAT) and it doesn't do the operation asynchronously unless it finds all metadata needed to carry it out is currently cached
<netbsduser>
heat: very important for proper software, database engines and such
<heat>
i read all about an nt stan creaming himself over the nt kernel in hacker news, regarding IO and async IO and oh god oh yes IRPs
<heat>
netbsduser, Proper Software uses files
<immibis>
Isn't that stuff pretty good though
<immibis>
Like Linux barely has async IO at all (has some now with io_uring)
<netbsduser>
heat: maybe the ones that can afford not to deal in disk blocks themselves
<netbsduser>
oracle database can work directly on a partition
<heat>
they probably all can deal in disk blocks
<netbsduser>
as to windows IRPs they are mostly good but they suck in one respect (the dispatch functions in the device stack recursively call the next driver. not good enough. STREAMS by contrast is iterative and doesn't engorge itself on the stack)
<heat>
oracle db actually burdened linux with maintaining a SCSI api layer because "we totes need to send raw SCSI commands boss"
<heat>
unix is generally just better there because you don't need terrible abstractions
<Ermine>
STREAMS
<kof673>
eh, conway's law. in theory the more direct, the less unneeded overhead. but conversely, a more broad, higher level, approach may have more idea what other things are doing versus every program for itself. it does not surprise me a db would want that though
<kof673>
that is arguably how dbs started, the one application that the os was built to run
<kof673>
oracle is not that old, but .....that was surely the norm at one time
memset has quit [Remote host closed the connection]
memset has joined #osdev
Left_Turn has joined #osdev
Turn_Left has joined #osdev
Left_Turn has quit [Ping timeout: 252 seconds]
Left_Turn has joined #osdev
goliath has joined #osdev
Turn_Left has quit [Ping timeout: 252 seconds]
the_oz_ has quit [Quit: Leaving]
the_oz has joined #osdev
sortie has quit [Remote host closed the connection]
sortie has joined #osdev
goliath has quit [Quit: SIGSEGV]
bauen1 has quit [Ping timeout: 260 seconds]
heat has quit [Remote host closed the connection]
heat has joined #osdev
memset has quit [Remote host closed the connection]
memset has joined #osdev
<heat>
Ermine, STREAMS
_ngn has joined #osdev
xenos1984 has quit [Quit: Leaving.]
xenos1984 has joined #osdev
<sortie>
... and now for the fourth gcc compile. Is it going to go through?
<heat>
i vote maybe
voidah has quit [Ping timeout: 272 seconds]
<_ngn>
hello, i was just following the long mode article (https://wiki.osdev.org/Setting_Up_Long_Mode), in the paging section, article explains how to set up tables needed for paging, and it uses the memory address 0x1000 to store the tables and assumes that its free to use - is there an actual way to make sure an address is free to use tho?
<bslsk05>
wiki.osdev.org: Just a moment...
vinleod has quit [Quit: Life beckons]
<heat>
no, not really
<heat>
well, it is possible, but practically in assembly it's hard and a bad idea
<heat>
best idea is to just statically allocate those in your program image (in .bss) and use them that way
<_ngn>
but that would just make the binary larger for no reason
<_ngn>
no?
<Ermine>
not really
<GeDaMo>
bss doesn't take up space in the binary
<_ngn>
oh so the space only gets allocated when the binary gets loaded into the memory?
<GeDaMo>
Basically, yes
<heat>
yes
<immibis>
bss: BaSically yeS
<_ngn>
lol, ill use bss then, thank you guys
bauen1 has joined #osdev
<the_oz>
you could also ask the bios for free memory listings
<nikolar>
you even get a nice warning that you could damage your computer
<heat>
as to the why: 1) you were probably booted by a bootloader (GRUB?), which likely means you're in protected mode
<heat>
2) point 1 means BIOS calls don't work. you're out of luck
xenos1984 has quit [Ping timeout: 246 seconds]
<Ermine>
lol @ making point 2 to clarify point 1
<heat>
3) you can parse a bunch of shit (EFI? multiboot info?) in assembly. but, do you _really_ want to? why do you want to hardcode your initial page tables' addresses?
<heat>
4) 0x1000 isn't guaranteed to be available. lets say it's not. what now?
<heat>
5) do you _really_ want to write all of this crap in assembly?
frkazoid333 has quit [Ping timeout: 276 seconds]
<immibis>
you can call the bios from real or v86 mode?
<heat>
gosh
<Ermine>
going v86 to fetch memory map?
<heat>
6) fuck real mode, fuck v86 mode, fuck unreal mode, run
<Ermine>
x86 linux can go v86 iirc
<heat>
i think so yeah. but like, run, please no, oh god please no stop dont hurt me
<_ngn>
ok i found out multiboot2 actually passes those memory information after boot
<_ngn>
so i can actually access it without calling the interrupt
<heat>
ok now read all of the other points
voidah has joined #osdev
<heat>
all of whatever logic you're writing can easily be replaced with: .section .bss\n pml4:. skip 4096\n pdpt: .skip 4096\n <...>
<heat>
6 LOC of stupid simple assembly. heck, C would work
<_ngn>
i mean the reason im doing all this is just to get better at assembly
<_ngn>
and bss sounds like a "hacky way" to get around this problem
<heat>
you'll have plenty of opportunities to write assembly later
<heat>
it's the proper way
<heat>
linux uses .bss
<_ngn>
how is the proper way
<heat>
bss is designed for exactly this
<heat>
large amounts of zeroed data that aren't supposed to occupy disk space and you can easily Just Use
<_ngn>
its designed for storing uninitialized static variables, no?
<heat>
what do you think the page tables are?
<heat>
they aren't even particularly used, you'll discard them like... 10 or 20 function calls into the kernel
<_ngn>
i dont think an entire program stack or page table entries count as static variables
<heat>
well, you're wrong
<_ngn>
wdym discard them
<heat>
you'll inevitably want to rebuild them later at boot in some shape or form because bootstrapping page tables in assembly is hard and everyone kind of halfasses it
<immibis>
it doesn't matter what you put in it. It's zero initialized data space.
<immibis>
You could use the memory right after your kernel, but guess what, that's what bss does
<heat>
yes. even conceptually, the first UNIX kernels didn't distinguish between .bss and the heap. it was just "the data area"
<heat>
malloc was basically a way to dynamically grow .bss :)
<immibis>
That's brk/sbrk
<nikolar>
(which malloc used in background)
<immibis>
btw malloc still isn't very good at reclaiming memory, so maybe we haven't come so far from brk/sbrk
<immibis>
it's inherent to the design of malloc/free that it doesn't know what goes where, so if you allocate a bunch of memory and free half, you're likely to end up with a bunch of half free pages, instead of half the pages free, and nothing can be unmapped
<_ngn>
afaik malloc only allocates a new page if the current page is fully used
<immibis>
it's possible that your program simulates a memory leak even if you free everything you malloc
xenos1984 has joined #osdev
<nikolar>
except that if you keep allocating and freeing the same sized allocations, you'll be reusing those holes and everyone's happy
<heat>
immibis, depends on how your malloc works
<heat>
also: MADV_DONTNEED
<heat>
you don't need munmap to reclaim memory at all
<heat>
AFAIK glibc malloc is actually pretty aggressive in MADV_DONTNEED'ing memory
gog has joined #osdev
<immibis>
that doesn't help the fact you have a bunch of pages with only 1 allocation on them
<nikolar>
that's basically impossible to happen
Arthuria has quit [Ping timeout: 276 seconds]
<the_oz>
If you're using GRUB then of course you don't want to ask the BIOS for a listing of free memory. Isn't there a way to get GRUB to do it for you? Whatever. You wanted a way to know and the way is told. Up to you if running some boot code is too close to the metal in *checks title* writing an OS...
<the_oz>
Furthermore, manual probing is NOT asking a BIOS
frkzoid has joined #osdev
tru3humandesign has joined #osdev
netbsduser has quit [Ping timeout: 272 seconds]
Dead_Bush_Sanpai has quit [Read error: Connection reset by peer]
Dead_Bush_Sanpai has joined #osdev
Dead_Bush_Sanpai has quit [Read error: Connection reset by peer]
Dead_Bush_Sanpai has joined #osdev
netbsduser has joined #osdev
voidah has quit [Remote host closed the connection]
voidah has joined #osdev
voidah has quit [Remote host closed the connection]
voidah has joined #osdev
<sortie>
heat: build/genautomata out of memory'd. It was the drop that pushed it over.
<sortie>
Let's add another gig to the build vm
<sortie>
fuck it, two.
<sortie>
(It's because I do the native builds in the ramfs rather than the ext2 filesystem because it's much, much faster)
<bslsk05>
stackoverflow.com: c++ - How to query BIOS using GRUB? - Stack Overflow
<sortie>
The memory map with GRUB is easy. You implement the multiboot specification, it passes you a pointer to a data structure in a register, that data structure contains the memory map for you. Easy peasy.
<sortie>
Do not go calling the BIOS if you're booting with GRUB. It's much easier and more reliable to get the information via multiboot 1 or 2. It also works on EFI.
<the_oz>
yes
<heat>
sortie, what if we had memory but not really
<heat>
imaginary memory
<sortie>
heat: Curiously that is what qemu transparently does
<heat>
you mean the kernel
<heat>
qemu does not swap
<sortie>
Unclear if qemu mmap's before the memory is touched the first time
<heat>
it does
<heat>
nikolar, that worst case is practically impossible yeah, but reclaiming heap pages is a real problem (particularly when you don't have access to shit like MADV_DONTNEED, like in the kernel)
<heat>
but alas memory defragging is really hard sooOOOOooooooooOOOOOoo
tru3humandesign has quit [Quit: WeeChat 4.4.1]
memset has quit [Remote host closed the connection]
theyneversleep has quit [Remote host closed the connection]
<netbsduser>
_ngn: your preferred bootloader will pass you a memory map
* Ermine
is ready to whine about setuptools again
<_ngn>
yeah i figured that out
<_ngn>
but i got bullied by .bss fans
voidah has quit [Remote host closed the connection]
voidah has joined #osdev
alexander has quit [Quit: ZNC 1.8.2+deb3.1+deb12u1 - https://znc.in]
alexander has joined #osdev
X-Scale has joined #osdev
craigo has joined #osdev
goliath has joined #osdev
X-Scale has quit [Ping timeout: 256 seconds]
<immibis>
it makes sense. why search through the memory map to find a place you like, instead of letting the compiler pick one? (really the compiler-chosen address is just a hardcoded address too, but it automatically hardcodes one that doesn't conflict with your kernel, and you only have to worry about putting the whole kernel block in a valid place in the memory map, such as 0x00100000)
GeDaMo has quit [Quit: 0wt 0f v0w3ls.]
jedesa has joined #osdev
foudfou has quit [Ping timeout: 260 seconds]
foudfou has joined #osdev
hwpplayer1 has quit [Remote host closed the connection]
Left_Turn has joined #osdev
Turn_Left has quit [Ping timeout: 244 seconds]
youcai has quit [Ping timeout: 276 seconds]
voidah has quit [Ping timeout: 255 seconds]
memset has quit [Remote host closed the connection]
memset has joined #osdev
youcai has joined #osdev
<sortie>
Alright that should port binutils-2.43.1 to my OS :D
<nikolar>
nice
steelswords has quit [Read error: Connection reset by peer]
steelswords has joined #osdev
levitating has joined #osdev
goliath has quit [Quit: SIGSEGV]
Turn_Left has joined #osdev
<kof673>
well...i'm sure there are binary formats where it literally is just a ram/rom dump, zeroes and all or whatever else .....any more advanced object format i assume will have something like .bss
<kof673>
"embedded" type stuff maybe
Left_Turn has quit [Ping timeout: 265 seconds]
<heat>
a.out was pretty stupid but still had .bss
<heat>
and flat binary accidentally does .bss (without the clearing, but that can trivially be done by startup code)
mavhq has quit [Remote host closed the connection]
XgF has quit [Ping timeout: 260 seconds]
mavhq has joined #osdev
asarandi has quit [Quit: WeeChat 4.2.2]
asarandi has joined #osdev
netbsduser has quit [Ping timeout: 248 seconds]
foudfou has quit [Remote host closed the connection]