<Clockface>
i figured out that the somewhat unusual/braindead way of doing things mean i can write mine to run in the userspace of a real operating system if i swap out the bootloader and drivers
biblio has quit [Quit: Leaving]
<Clockface>
well, part of it at least
<Clockface>
its enough that it makes my life a hell of a lot easier
dude12312414 has quit [Quit: THE RAM IS TOO DAMN HIGH]
sonny has quit [Ping timeout: 240 seconds]
_xor has quit [Quit: brb]
_xor has joined #osdev
Vercas has quit [Remote host closed the connection]
Vercas has joined #osdev
ephemer0l has joined #osdev
nyah has quit [Ping timeout: 250 seconds]
EtherNet has quit [Ping timeout: 240 seconds]
EtherNet has joined #osdev
sdfgsdfg has joined #osdev
ElectronApps has joined #osdev
Clockface has quit [Ping timeout: 240 seconds]
sdfgsdfg has quit [Quit: ZzzZ]
Vercas has quit [Remote host closed the connection]
Vercas has joined #osdev
sonny has joined #osdev
ElectronApps has quit [Remote host closed the connection]
sonny has quit [Quit: Going offline, see ya! (www.adiirc.com)]
kingoffrance has quit [Ping timeout: 240 seconds]
[_] has joined #osdev
[itchyjunk] has quit [Ping timeout: 250 seconds]
[_] is now known as [itchyjunk]
ZombieChicken has joined #osdev
ZombieChicken has quit [Ping timeout: 276 seconds]
Sos has quit [Quit: Leaving]
ElectronApps has joined #osdev
mahmutov has joined #osdev
sdfgsdfg has joined #osdev
ElectronApps has quit [Remote host closed the connection]
GeDaMo has joined #osdev
ElectronApps has joined #osdev
epony has joined #osdev
heat has joined #osdev
[itchyjunk] has quit [Read error: Connection reset by peer]
srjek has quit [Ping timeout: 240 seconds]
kingoffrance has joined #osdev
tenshi has joined #osdev
<kazinsal>
I just wrote a 3500 word forum post on the history and insanity of x86 memory management because a bunch of folks liked my (much shorter) effortposts about the VAX and I feel intellectually beat after explaining the history of Intel's "just pile it on, it'll be fine" architecture
biblio has joined #osdev
<geist>
oh wow
<kazinsal>
people were like "man that's more effort than we were expecting about VAX, what else you got"
<kazinsal>
their mistake
Jari-- has quit [Remote host closed the connection]
heat has quit [Ping timeout: 250 seconds]
<geist>
kazinsal: is there a link to this discussion?
<kazinsal>
depends on if you have a SomethingAwful forums account or not, pretty sure YOSPOS is behind the paywall
<geist>
ah oh well
<kazinsal>
it started because someone posted a thread called "tell me about the DEC Alpha" and then someone asked VAX and I chimed in
<geist>
i never 100% got clear on precisely how the MMU worked on the alpha
<geist>
ie, is it a pure software TLB + software routines implemented in PALcode?
<geist>
(i think thats the answer)
<kazinsal>
so then the thread got renamed to "tell me about DEC hardware" and I joked about writing something on the x86's bizarre memory management and a few people were like "oh god please do"
<geist>
and the standard PALcode implements basically a standard page table
<froggey>
geist: from the little I poked around with Alpha, that's exactly it
<geist>
yah. i wonder if the 2 or 3 standard palcode libraries implemented it the smae way?
<geist>
i understand there was a DEC one for VMS and unix and then there was the one for NT
<kazinsal>
probably. I think raymond chen wrote a bunch of stuff about NT on Alpha, let me dig that up
<froggey>
dunno. I know that the source for at least one palcode is available
<geist>
wouldn't be surprised in the NT one was nerfed to be 32bit (since NT ran in 32bit mode) and thus used 4 byte PTEs etc
<geist>
froggey: yah i guess it was set up such that you can configure the TLB miss exception to directly vector to PAL code
<geist>
or maybe regular exceptions are always run after at least one instruction of palcode
<geist>
and the whole exception delivery mechanism is itself implemented in palcode
<geist>
has a certain 'microcode all the way down' feel to it
<froggey>
yeah, the cpu vectors directly to a fixed palcode entry and the palcode is what signals any kind of "regular" exception
<kazinsal>
"An Alpha AXP program running on 32-bit Windows NT still has full access to the 64-bit registers and can use them to perform 64-bit computations. It could even use the full 64-bit address space, if you were willing to jump through some hoops."
<kazinsal>
neat
<froggey>
iirc all exceptions & interrupts went to palcode and it was responsible for implementing regular exceptions
<geist>
i haven't looked into qemu alpha support but i wonder if qemu attempts to run palcode, or if it emulates it as if it already had palcode loaded...
<froggey>
fun fact: the last model of lisp machine sold by symbolics was an alpha axp workstation running an Ivory emulator
<froggey>
running on stock hardware ended up being cheaper/faster than designing custom hardware
<geist>
neat
<geist>
the one time i ever used an alpha workstation it was a screamer
<geist>
i had a little DEC Multia years later, after it was obsolete
<geist>
alpha hardware is still hyper expensive on ebay, must be a real demand for them in production
<nomagno>
geist: I gave up and used a FLOSS alpha VM made by an angry ex-employee, it was buggy but good enough to do the OpenVMS testing I needed
<nomagno>
OpemVMS is getting an x86 port, apparently
ElectronApps has quit [Remote host closed the connection]
sdfgsdfg has quit [Quit: ZzzZ]
pretty_dumm_guy has joined #osdev
lkurusa has joined #osdev
Tangent-Man has joined #osdev
lkurusa has quit [Client Quit]
lkurusa has joined #osdev
lkurusa has quit [Ping timeout: 240 seconds]
<geist>
yah i think it's in some sort of beta and/or released now
Jari-- has joined #osdev
biblio has quit [Quit: Leaving]
Burgundy has joined #osdev
simpl_e has joined #osdev
klange_ is now known as klange
nyah has joined #osdev
NeoCron has joined #osdev
C-Man has joined #osdev
mahmutov has quit [Ping timeout: 240 seconds]
mahmutov has joined #osdev
dude12312414 has joined #osdev
<Griwes_>
geist: what's the main point behind the hierarchical design for VMARs in zircon? I'm having trouble imagining a situation where I'd want to create a child VMAR and lower its permissions, since for the cases that come to mind, the creation of it would be pretty much right next to the call to zx_vmar_map. Is there some cases of foreign processes wanting to map something into your address space and needing (from the non-foreign perspective) to have lowe
<Griwes_>
permissions that I'm missing?
Griwes_ is now known as Griwes
terminalpusher has joined #osdev
freakazoid333 has quit [Ping timeout: 240 seconds]
dude12312414 has quit [Remote host closed the connection]
dude12312414 has joined #osdev
srjek has joined #osdev
Bonstra has quit [Quit: Pouf c'est tout !]
Bonstra has joined #osdev
<nomagno>
Is there a generic term for a program that is both an assembler and a bytecode interpreter?
<sortie>
interpreter?
srjek has quit [Ping timeout: 240 seconds]
terminalpusher has quit [Remote host closed the connection]
freakazoid333 has joined #osdev
ckie has quit [Quit: off I go~]
ckie has joined #osdev
zaquest has quit [Remote host closed the connection]
zaquest has joined #osdev
sonny has joined #osdev
PapaFrog has quit [Ping timeout: 256 seconds]
PapaFrog has joined #osdev
pretty_dumm_guy has quit [Quit: WeeChat 3.4]
pretty_dumm_guy has joined #osdev
mahmutov has quit [Ping timeout: 240 seconds]
sonny has quit [Ping timeout: 240 seconds]
sonny has joined #osdev
<Ermine>
Translator?
dude12312414 has quit [Quit: THE RAM IS TOO DAMN HIGH]
<nomagno>
I ended up splitting the interface into 'assembler' and 'exec'
GeDaMo has quit [Remote host closed the connection]
sonny has quit [Ping timeout: 256 seconds]
sonny has joined #osdev
sonny has quit [Ping timeout: 256 seconds]
sortie has quit [Quit: Leaving]
sortie has joined #osdev
tusko has joined #osdev
<geist>
Griwes: whether or not you lower the permissions, each VMAR (and sub vmar) has a handle (or no handle, you can throw away the last copy of it) and thus only users with the handle can operate
<geist>
so for example you can carve off the frst 4GB of the address space and give it to the loader, and then no other code has that handle and thus cannot manipulate anything in the 0-4GB range
<geist>
or you can create a sub vmar + handle and give it to another process and they can then only manipulate that range of the address space
<geist>
permission wise yes the heirarchy of permissions is somewhat less used, also since the handle itself has a right which can additionally layer on top of it
<geist>
but you can for example create a range of your address space -X permission
<geist>
and then give the handle to anyone. they might create another sub vmar, but it would break the permission model if they could add +X to their child
<geist>
since it would defeat the purpose of having -X in the first place
<Griwes>
the first example you gave has nothing to do with *security*, right? because you must assume that whoever is in your address space can find that handle by probing memory?
<geist>
no. they cannot period
<geist>
the handle is private to the aspace, cannot be cloned externally, etc
<Griwes>
right, but zircon has an in-process loader, right?
<geist>
you cannot find a handle to a thing by probing. the only way to do it would be to have code in that process actually duplicate it
<Griwes>
so non-loader code could find that handle in memory?
<geist>
the handle doesn't exist in memory. you'd have to probe around the actually find it in the .data segment of the loader
<geist>
however there's also a situation where the loader actually throws the handle away
<geist>
one of the differences in VMARs to most zircon objects is throwing away the last handle to the vmar does not destroy t
<geist>
it stays in place and then that part of the aspace is basically locked down
<Griwes>
yeah I've read the docs ;p
<geist>
so you can also create a locked down procss by either externally loading it (and thus there is no local handle to the vmar where code is) or loading, and then discarding it
<geist>
but yes, you could sniff around and try to find the handle in the loaders data segment i think in the normal case where the handle is left around
<geist>
there is somewhat of a combination of external and in-process loaders for normal processes on fuchsia, so yes there is some general need for fiddling with the local aspace after it starts, for dyload() etc
<geist>
or at least i think that's what goes on
<Griwes>
heh
<Griwes>
alright, thanks for the commentary
<geist>
but yeah re: heirarchial permissions, it's mostly so you can't just create a sub vmar which raises the elevation, otherwise it defeats the whole purpose of having permissions on it
<Griwes>
I'm still trying to square kernel vs userspace vmm in my head
<Griwes>
currently I'm at the stage where I hate both
<Griwes>
:D
<geist>
on zircon or in general? the heirarchial VMAR stuff is AFAIK unique to zircon
<Griwes>
(well, "hate")
<geist>
most systems are flatter than that
<Griwes>
for my kernel
<geist>
when you say kernel vs usespace you mean implementing a VMM in onr or the other, or something like 'kernel responsibilities' vs 'userspace responsibilities'?
<Griwes>
yeah I'm thinking flat but I'm trying to figure things out and there appears to be no approach that I actually, actively, like
<Griwes>
implementing a VMM
<Griwes>
was it L4 that had a fully userspace one?
<geist>
ah i once implemented a kernel where the VMM was implemented in user space. was interesting, but didn't get far enough to really explore what it was going to look like
<geist>
in that case i still had the PMM implemented in kernel space. so it had some specialized syscalls the vmm would use: pmm_get_pages, pmm_put_pages and a bunch of page table calls: mmu_map_page, mmu_unmap_page, mmu_set_protection, etc
<geist>
had a way to register for page faults to be delivered into the VMM process, and then took it from there
<geist>
having the pmm in the kernel meant the kernel could still internally grab pages for it own use, but most of the list f free pages were consumed and returned by the userspace VMM
<geist>
then that meant all the usual VMM calls were IPC to the VMM process
<geist>
i think it would have worked okay
<tusko>
does anything about this syscall and test seem clearly incorrect to you, according to std. practices? https://bpa.st/VEXA
<bslsk05>
bpa.st: View paste VEXA
<geist>
oh also there were some syscalls for mmu_create_aspace -> handle to aspace, mmu_delete_aspace, etc
Tangent-Man has quit [Remote host closed the connection]
<geist>
and as processes were created in user space they'd call into the VMM to allocate a fresh handle to associate with a process
<geist>
something like
<geist>
create_process -> proc_handle, create_aspace -> aspacE_handle, bind_apsace_to_process(process_h, aspace_h) as part of one of the steps of process creation
<geist>
(this was a µkernel)
<geist>
for zircon sicne the IPC is asynchronous we decided to just leave the VMM in the kernel as well
<Griwes>
yeah, that makes sense
<geist>
the µkernel i was talkig abot where VMM was in user space was fully synchrnonous, so the kernel only needed to allocate memory for its own purposes when you created/freed objects, or mapped pages (for page tables)
<geist>
otherwise it didn't really consume anything from the pmm
<Griwes>
user-space vmm has silly problems like "how does the syscalls mechanism verify the vdso locations" or "how do I manage permissions for the mmu syscalls"
<Griwes>
and I've been spinning on trying to make a decision for quite some time now
<j`ey>
heh, linux 3.9 defconfig took 6mins to build, my minimal config for 5.16 takes 30mins :/
<Griwes>
(like, before I build something non-trivial in the kernel or whatnot)
<geist>
Griwes: well simplest one for the VMM permission is to simply have some priviledged 'i'm a superserver' bit set on the process and move on
<Griwes>
yes
<geist>
the more capability based one would be to have some sort of priviledged handle to the PMM/MMU kernel feature that the VMM has to pass on every syscall
<Griwes>
but that raises further questions of "can I have more than one of these?"
<geist>
we cheated a bit on zircon by having a 'resource' handle that unlocks high end permissions for things like that
<Griwes>
"can different processes select different policies by talking to different vmms?"
<geist>
sure. that'd be easy since you could creat ean aspace object
<geist>
how about create_aspace -> aspace_handle
<geist>
bind_aspace_to_vmm(aspace_handle, handle to page fault IPC/etc)
<Griwes>
right; I'm not saying it's impossible, it's just that it raises so many questions that I'd need to find at least draft answers to to be comfortable with going that way
<geist>
now every time you create an apsace you bind it such that all future page faults, etc get delivered to the IPC/etc channel the VMM has
<Griwes>
I like to understand the consequences of choices I'm making and the decision tree here is so damn deep
<geist>
that way if some other thing wanted to take on that responsibility the kernel doesn't care
<geist>
i think the key is always think in a capability like way: to create a Thing you must have a handle to Another Thing
<geist>
and then you can have operations that glue two or more things together, provided you have handles to it
<geist>
obviously there's bootstrapping issues, and the first process probably cheats and gets handles to Uber Things delivered to it by the kernel, but then user space chooses how to subdivide responsibility
<Griwes>
right
<Griwes>
but the first process needs handles delivered to it anyway
<Griwes>
so at least that part is not a big deal
<geist>
in the aspace handle case you now have a capability to lock off the mmu_map/unmap calls
<geist>
mmu_map_page(aspace_handle, ...)
<geist>
the only real magic capability yo have there is the ability to create the aspace in the first place.
<Griwes>
right; that's probably managed by the same capability that allows creating new processes
<geist>
also if you keep aspace handles and process handles separate you can do some fun stuff like have two processes use the same aspace
<Griwes>
or at least something of a similar magic level
<geist>
or have aspaces with things mapped that no process is using
<Griwes>
I really really don't want to have two processes use the same vas
<geist>
or some way to create a prototype aspace that maybe you add some sort of aspace_clone() call
<Griwes>
but yeah having a detached vas seems reasonable
<geist>
or maybe that's exactly the way you create new aspaces: clone an existing, empty one
<geist>
and thus you have the capability to make new aspaces by the idea that you have an existing empty, clonable handle
<geist>
the 'create new things by cloning existing things only' is also an interesting route to go down. we ddn't take that route in zircon
<geist>
but if you explore it you can also come up with interesting patterns
<Griwes>
yeah
<Griwes>
alright, I have plenty of food for thought now, thanks!
<geist>
but still it's fundamentally a capability style route
<geist>
sure thing!
<geist>
i thin kthe trick is tocome up with some paradigm and then try to map it to everything so you end up with a consistent pattern
<geist>
you might have to make exceptions here or there, but its still a nice design if you can mostly build something consistent
* Griwes
nods
<tusko>
please to help tusko :(
<tusko>
i will be your friend forever
sdfgsdfg has joined #osdev
<Griwes>
Alright I think this discussion has pushed me into a specific direction, which means I can get unblocked on actually writing OS code
<geist>
Griwes: yay!
<geist>
tusko: okay, hang on, moving to another room will take a look
<tusko>
thank you so much!
<geist>
tusko: in general you dont need to copy from user since hte output is overwritten
<geist>
otherwise looks okay. not sure if you have anything in particular you want a commentary on
<tusko>
I guess it just didn't work when I compiled it
<tusko>
I was concerned about issues of isolation between userspace memory and kernel memory
<tusko>
That test program reports back 1 + 2 is 0
<geist>
well, i dont know exactly what it takes to add a linux syscall, which is what this appears to be
<geist>
but the overall strategy of what you're doing here looks okay
<geist>
if you want actual 'how do i do this in linux' this isn't really the right channel
<geist>
as an overall 'how syscalls work generically on kernels' this seems reasonable
<geist>
like in this case is the result pointer actually pointing to user space? duno. i'd try printing it
<geist>
also make sure the arg direction is correct in copy_to_user
<tusko>
Ok. Yeah, I'm not perfectly sure either. I definitely have a handle on how to add it to the kernel, but the mechanics of passing that pointer around and writing it are a little opaque to me
immibis has joined #osdev
<geist>
yah might be intersting to see what the absolute value of the pointer is
<geist>
maybe there's some layer above in the kernel that makes a temporary for you that result is actually pointing at?
<geist>
honestly the fact that it has naked pointers to stuff in user space is pretty scary
<geist>
seems so easy to accidentally violate the rules of not touching things directly in user space
<tusko>
I think I was actually missing a header <linux/uaccess.h> also. I saw some errors I hadn't noticed before while building about implicit definition of those copy functions
<tusko>
its pretty shitty having to wait like 30m per compile. how do you cope as an OS developer?
<geist>
oh eep. yeah fix the warnings
sonny has joined #osdev
<tusko>
that seems to have done it. I also changed the copy calls to copy_from_user(&tmp, result, sizeof(int)) and copy_to_user(result, &tmp, sizeof(int)) respectively
sdfgsdfg has quit [Quit: ZzzZ]
tusko has quit [Remote host closed the connection]
tusko has joined #osdev
jjuran has quit [Ping timeout: 240 seconds]
srjek has joined #osdev
jjuran has joined #osdev
jjuran has quit [Remote host closed the connection]
jjuran has joined #osdev
flx has quit [Ping timeout: 256 seconds]
tusko has left #osdev [Byeee]
tusko has joined #osdev
ZombieChicken has joined #osdev
heat has joined #osdev
<heat>
hi sortie
<sortie>
Jeez the heat is onto me
<heat>
do you have any advice on porting packages that use cmake? without the environment leaking
<sortie>
This is a the winning move is not to play type situation and I won
<heat>
i've wrapped pkg-config but I see there's still some weirdness with the way cmake finds packages and the toolchain file
<heat>
dunno if you've seen stuff like this
<sortie>
Would you believe I literally don't have any ports using cmake?
<heat>
wowzers
<sortie>
So I don't know much about cmake but I get the impression a lot of the interface and logic is offloaded to people's cmake scripts so the behavior is probably going to vary wildly to the point that it's not a standard interface?
<sortie>
(Where ./configure is a fairly standardized interface although ports sometimes do not quite play by the rules and I fix that)
<heat>
hmmm not so much, cmake has a bunch of defined interfaces to find packages and libraries and whatnot
<heat>
like budget pkg-config
<sortie>
What problematic behavior exactly are you seeing with cmake?
<heat>
and cross compilation is pretty well defined
<heat>
the host environment is leaking
<heat>
I think I found out why but i'm not too sure
<graphitemaster>
you just chroot to build every package
<graphitemaster>
that's the only sane way
<heat>
issue: I'm building the least trivial package, llvm
<sortie>
The general debugging pattern is to find a value that's wrong and try and trace it back to where it was determined, and see at what stage there's a confusion between build/host/target
<heat>
from a quick google before I went to bed last night I think I found out that cmake is really stupid when find_package() is involved
<heat>
like, it uses your sysroot but if it can't find the package it tries the host libraries *for some reason*
<sortie>
graphitemaster, I hear you, though that's also coming from a perspective of “oh man gotta herd all these upstream cats”, and the alternative philosophy I take is to nail down all of these cross-compilation bugs (and they are bugs, conceptual bugs, not using the right variables, doing wrong stuff, etc.) and actually fixing them
<sortie>
So yeah I gotta get why de-facto Linux distros would want to chroot such builds because stuff is fucked yo