<TwoNotes>
Is there an example somewhere of using the mstatus.mprv bit to do address translation in M-mode?
<jrtc27>
bbl and opensbi
iorem has quit [Quit: Connection closed]
vagrantc has joined #riscv
<TwoNotes>
It looks like MPRV lets me do a single fetch or store, but not get the address of, say, an I/O buffer in user space. For that I need to write a software address translator.
<TwoNotes>
BBL may already contain such a hting
<TwoNotes>
For example, U-mode does an ecall passing the (virtual) address of a buffer to be filled. How to get physical address of that buffer in Supervisor mode?
<TwoNotes>
or even in Machine mode
FluffyMask has joined #riscv
sm2n_ is now known as sm2n
jebba has quit [Quit: Leaving.]
<jrtc27>
TwoNotes: you're asking the wrong question
<jrtc27>
the question is not "how do I get the physical address", it's "how do I load via a virtual address"
<jrtc27>
which is... what MPRV does
<jrtc27>
in a sense
<jrtc27>
if MPP is S (01) and MPRV is 1 then it's as if S-mode made the access
<jrtc27>
if you *really* need the physical address in M-mode then you have to walk the page table yourself
<jrtc27>
but I highly doubt you do
<jrtc27>
and it's a very dangerous thing to use for anything non-trivial
<jrtc27>
(what happens if you walk the page table checking for read permission but then write to it? (the write is permitted even if the page is read-only, which causes all manner of problems, not just security ones.) what happens if you read past the end of the page? (you get the next physical page, which is probably not the same as the physical page for the next virtual page))
Gravis has quit [Read error: Connection reset by peer]
Gravis has joined #riscv
Gravis has quit [Ping timeout: 252 seconds]
Gravis has joined #riscv
SwitchToFreenode has joined #riscv
<xentrac>
jrtc27: if you need to send the physical address to some DMA device so it can DMA into memory, which is what you'd normally want to do with an I/O buffer, you need to get the physical ddress, not loada via a virtual address
<jrtc27>
yes
<jrtc27>
this is not a thing M-mode should be doing
<xentrac>
who should be doing it then?
<xentrac>
or do you mean people shouldn't use DMA?
<jrtc27>
S-mode
<jrtc27>
U-mode calling into M-mode to do DMA screams of "no god no"
<xentrac>
I think how to do it in S-mode was TwoNotes's original question?
<xentrac>
then they appended "or even in Machine mode"
<xentrac>
oh, no, before that they were asking about M-mode
<jrtc27>
oh that was buried in the middle, after initially talking about M-mode and then MPRV
<xentrac>
you're right
<jrtc27>
the question is far too vague and confused to know what they really want quite frankyl
<jrtc27>
because the things M-mode and S-mode want to do are mostly disjoint
<jrtc27>
(and normally in S-mode you have data structures on the side describing the current address space...)
<xentrac>
i think they want to write a device driver for a DMA device that can communicate directly with a user-mode process without the cost of context-switching on traps
<xentrac>
like VIA for Ethernet
<xentrac>
but I admit that's just a guess
<jrtc27>
in that case... kernel should just look up what pages it handed out to userspace for the mapping userspace is trying to use
<jrtc27>
*and make sure to pin them*
<xentrac>
heh, yeah
<xentrac>
seems like a reasonable kind of thing to do for low latency, doesn't it?
<jrtc27>
sure, in that niche market
<jrtc27>
but you need to be very careful
<jrtc27>
and if you're asking *those* kinds of questions you really should not be the person to write that code
<xentrac>
is that a niche market? I feel like arduino is kind of popular actually
<jrtc27>
who's running unix on an arduino
<xentrac>
yeah, virtual memory is not a great fit for submicrosecond latency
<xentrac>
but it's not unheard of. I mean that was almost Data General's entire market
<xentrac>
although I guess at the time it wasn't submicrosecond ;)
<xentrac>
and you have lots of stuff like linuxcnc and vxworks
cmuellner has quit [Ping timeout: 272 seconds]
Thalheim has joined #riscv
Gravis has quit [Ping timeout: 272 seconds]
Gravis has joined #riscv
<TwoNotes>
Device drivers go in Smode generally.
<TwoNotes>
Cant the memory map be different between S and U modes?
<TwoNotes>
The larger U74-type hardware is not really for 'embedded" applications
<jrtc27>
different between S and U> yes and no
<jrtc27>
the mappings are the same, but each page can be configured as either S or U mode
<enthusi>
leah2: so how's the first impression? :)
<leah2>
i hate the fucking gnu toolchains
<leah2>
but yeah, the board is fine. i'm getting a replacement fan on tuesday
<enthusi>
an expensive one of some cheap 12v 40mm one?
<enthusi>
the one someone posted goes for ~ 50 eur
<TwoNotes>
The other use for knowing the physical address is inter-process comm, which is usually a Machine-level thing, no?
<TwoNotes>
But I think looking up an adress in my own assignment tables is easier than trying to simulate the hardware Page Tables
<sorear>
IPC is handled at supervisor level in all of the current systems
<sorear>
moreover, since S-mode has access to translation tables, S-mode can phrase its requests to M-mode in terms of physical addresses
<sorear>
principle of least privilege dictates you do so; run all translation in S-mode
<TwoNotes>
Doesns seL4 use M-mode for IPC?
<sorear>
(if M-mode is enforcing a security boundary it needs to run range checks against addresses it receives equivalent to what PMP checking does, but this is simpler than a page table walk)
<sorear>
no part of seL4 runs in M-mode
<TwoNotes>
The reason I ask these questions is I am trying to learn the principles by building one from scratch, not just downloading Linux
<sorear>
(it *could*, since unique among modern kernels seL4 does not use kernel virtual addresses at all, but gernot fully supports the "M-mode is for instruction emulation only" paradigm)
<TwoNotes>
Boot runs in M. And sets up interrupt handlers. I suppose not much else HAS to
davidlt has quit [Ping timeout: 245 seconds]
<jn>
(to some confusion, IPC can mean "inter-process communication" or "inter-processor communication", depending on the context)
<TwoNotes>
In a multi-core system, user code is not supposed ot care
smartin has quit [Quit: smartin]
<geist>
TwoNotes: in general the idea is only basic firmware lives in M mode once it has finished booting. the OS and all of it's view of memory and translations are all S mode
<geist>
M mode just hosts some SBI firmware at that point, which just does very basic operations (flushing the tlbs, setting timers, etc) but it runs entirely in physical address space
<geist>
also note both M mode and S mode have their own interrupt and vector tables. some are trapped into M mode first and then reflected down (like the timer) and some go directly to S mode (most exceptions like page faults or whatnot) if they happened there
<TwoNotes>
So S mode can be trusted to set up its own memory maps to device registers, allocate physical memory, etc.
<geist>
right
<geist>
note that it can't touch the physical block of ram that the M mode firmware is running in because M mode has used a PMP to 'carve it off'
<TwoNotes>
seL4 provides mechanisms to be more restrictive than that, but you don't HAVE to use them
<geist>
so the firmware in M is still protected from S stomping on it
<TwoNotes>
It is certainly easier conceptually to only deal with two levels, S and U, most of the time
<geist>
(on qemu virt machine, for example the SBI firmware in M mode grabs the first 256KB of ram at 0x8000.0000 and keeps for itself. everything else is left to S)
<sorear>
important: not all systems have a security boundary between M and S
<jrtc27>
tbh the use of the pmp for opensbi is silly
<jrtc27>
it's more of a debugging aid than security
<jrtc27>
there is no threat model where it makes sense
<TwoNotes>
pmp = Physical Memory Protection?
<jrtc27>
(and can be trivially bypassed if you have a dma engine anywhere on your processor)
<geist>
yah, it's more of a freebie, 'may as well' (SBI using PMP for itself)
<jrtc27>
(uh, soc)
<geist>
in the situation where there's a single OS running and no sort of hypervisor or trusted firmware anywhere, it is indeed silly for M mode firmware (SBI in this case) to protect itself from the One OS running
<geist>
where it starts getting interesting is when you have multiple OSes (hypervisors, etc) or some sort of OS + trusted environment (which i dont think has been specced out on riscv yet/ever)
<jrtc27>
if there's a hypervisor then 2-stage translation protects the hypervisor and m-mode
<geist>
but not to overly complicate the answer, that's a bigger topic
<jrtc27>
the pmp adds nothing
<jrtc27>
unless you have much stronger platform support than riscv currently provides
<geist>
right. that's my point. it's very much almost exactly what arm64 already does (4 EL levels, etc) and the top level (EL3 which is directly analogous to M mode) can be used to switch between secure and insecure modes
<geist>
but... i dont think riscv seems to be going in that directly (fine with me!)
<jrtc27>
sure, but arm has much more support for that
<geist>
right
<jrtc27>
riscv tries to but is comically lacking for it to be at all meaningful
<geist>
sure. just sayin the logical extension if riscv were to go that way there's already a prior pattern
<geist>
if it were to go that way PMP would be nicer than the ARM solution of protecting EL3 from the lower rings which is 'yolo soc does it'
<sorear>
the PMP is there to support PMSA-type applications
<TwoNotes>
I thought I read somewhere in the specs that they are pulling back a bit from some of the "H" mode discussions
<jrtc27>
H mode got abandoned ages ago
<jrtc27>
H extension did not
<jrtc27>
it's a type 1 vs type 2 hypervisor discussion
<jrtc27>
making type 1 hypervisors fit in a type 2 world is easy
<jrtc27>
making type 2 hypervisors fit in a type 1 world is not
<jrtc27>
and effectively requires duplicating a whole bunch of code
<xentrac>
if M-mode vs. S-mode isn't enforcing a security boundary, can't you just run your instruction emulation traps in the same mode as your supervisor?
<jrtc27>
and either choosing which copy to use at run time, or having two different kernel builds depending on whether you're running natively on a machine with virtualisation support, or without virtualisation support (ie either on hardware without it or inside a VM)
<jrtc27>
which is a big fat no from both Linux and FreeBSD
<jrtc27>
c.f. AArch64 that started down the type 1 route then added a system configuration bit that switches it to type 2
<TwoNotes>
The architecture provides for "delegation" of trap handlers, so you could have M-mode hand off everything so S mode
Bigcheese has quit [Ping timeout: 248 seconds]
<jrtc27>
xentrac: for some of them you can, but some of them require access to M-mode state, so you might as well handle them in M-mode rather than take the overhead of bouncing them to S-mode and re-decoding
<xentrac>
I wish people would stop naming things "type 1" and "type 2". it's like a boolean function argument
<xentrac>
openFiles(FALSE);
<sorear>
xentrac: yes, and linux and bsd both have floating-point emulators, but it's a N*M problem
<xentrac>
sorear: ugh
Bigcheese has joined #riscv
<xentrac>
I guess it's too late for me to go bitch at Popek and Goldberg about their nomenclature, they're both dead
<sorear>
i don't see why this should stop you
<geist>
xentrac: yeah i still use type 1 and type 2, but really should quit it. it just Sounds Cool
<geist>
but modern hardware blurs the lines so much it's not really useful at all
<xentrac>
if I'm not the only one who had to look this up, type-1 hypervisors are bare-metal hypervisors, and type-2 hypervisors are the "hosted" ones that run under some other OS
<geist>
i think there's a teensy bit of value calling out a Hard Type 1 vs Everything Else
<geist>
and there are not that many hard type 1s out there
<geist>
OTOH, not sure the value in it except as a classification for marketing purposes
<jrtc27>
also, are vmm/kvm really type 2? they run bare-metal, really, but part of an OS that's also bare-metal
<xentrac>
I think KVM turns Linux into a bare-metal hypervisor but obviously I'm no expert ;)
<geist>
jrtc27: right that's where it's all blurry. i generally personally classify it based on whether or not the hypervisor itself is specifically written to just do that
<geist>
ie, ESXi, xen
<jrtc27>
yeah
<jrtc27>
"is there a user mode immediately above me" is my mental test
<xentrac>
right
<xentrac>
I feel like paravirtualization is another kind of gray area
<geist>
but then of course even for systems like ESXi, it has it's own user space, and it's own posix emulation, et
<dh`>
there are just layers of software, there's no meaningful distinction between kernels and hypervisors
<geist>
so it's kinda more like a tailored OS that is designed to mostly do one thing
<xentrac>
where you compile your guest OS to carefully refrain from invoking any privileged instructions
<jrtc27>
yeah never been a fan of "xen kernels"
<xentrac>
dh`: I think there is. a hypervisor is a kernel whose user processes can run without a kernel
<geist>
i suppose whatever the firmware IBM runs on their big Z machines or POWER based things are pretty close to hard pure type 1s but then no one sees the firmware so dunno what they 'look like'
<xentrac>
jrtc27: right, but is paravirtualization-era Xen a hypervisor or not? it's kind of blurry
<dh`>
xentrac: paravirt
<geist>
i dont see the two as orthogonal. i generally see paravirt as a fancy version of 'drivers tailored for running under this hypevisor'
<geist>
it's just a deeper cut to the guest
<jrtc27>
is it all that different from using EFI run-time services rather than poking the hardware yourself?
<xentrac>
geist: slightly further along that continuation you have ordinary Unix user processes
<xentrac>
dh`: ?
<xentrac>
*along that continuum
Narrat has joined #riscv
<geist>
sure. it's why it's all complicated, but in the end hardware gets designed to try to funnel designs into one spot
<geist>
hence x86 being like it is and being such a dominant player tending to drag things towards a specific design
<geist>
software and hardware each chase each others tail and you end up with some sort of local minima
<dh`>
xentrac: a paravirt hypervisor is not a kernel whose user processes can run without a kernel
<dh`>
also there are machines where the uppermost-level object is firmware and nothing else can run without it
<dh`>
e.g. sparc T1 (and I think T2 too)
<geist>
alpha i guess as well
<xentrac>
dh`: agreed, and I think that's an argument that paravirtualizing hypervisors aren't really hypervisors
<xentrac>
I mean UML was my first encounter with paravirtualization
<dh`>
my point was: it's all meaningless
<dh`>
you have software objects and they rely on certain sets of environment services
<dh`>
some look one way and we call them kernels, some look another way and we call them hypervisors, a lot look a third way and we call them applications or user programs
<xentrac>
I think different degrees of virtualization are a real thing that really does exist in the world though
<dh`>
others sit in the middle somewhere
<xentrac>
and virtualizability
<dh`>
e.g. microkernels are hypervisors done wrong, or vice versa
<geist>
that's kinda why i think the words are still meaningful, but only when describing things used in a particular context
<xentrac>
haha
<geist>
not the physical thing
<xentrac>
yeah
<geist>
ie, this isn't a rock today, it's a window smasher
<sorear>
there were ideas years ago that riscv would run in the "software-defined environment" direction similar to what dh` is talking about with sparc, but a lot of that ran into desire to build sub-MB embedded systems
<dh`>
the words are still meaningful, but only because they represent common concepts that are approximations of some real things
<xentrac>
it seems kind of goofy to argue that since FreeBSD can run Linux binaries, and UML is a Linux binaries, FreeBSD became a hypervisor once someone wrote UML
<geist>
<thumbs up> i've argued with folks that claim 'since i can't hard describe X with word Y word Y is meaningless and thus never use it again'
<geist>
which is not as useful to me
<xentrac>
sub-megabyte, sorear?
<jrtc27>
sorear: also, such systems only really work when there are only one or two vendors producing the hardware
<jrtc27>
when you have many, they all want to compete and give you fancy new devices that aren't abstracted away
<xentrac>
ugh, "is a Linux binary". can't brain
<dh`>
geist: there are a lot of people in tech who have very rigid approaches to reality
<geist>
dh`: yep
<xentrac>
s/ in tech//
<dh`>
some of it is probably neurodivergence but a lot of it I think is stupidity or poor education
<xentrac>
it's just the way humans are
<geist>
i'm perfectly happy living in an ambiguous world like that and then going a level deeper to see what it all *means*
<xentrac>
it's only that it's not obvious when you're living inside it
<jrtc27>
what's the alternative? even more vague emails from people reporting bugs? :P
<geist>
vague bug reporting is simply the state of the world. you try to improve it but it's always there :)
<xentrac>
the first noble truth
<dh`>
anyway
<dh`>
i would say that SBI protecting itself with a pmp entry is worthwhile, because it costs little or nothing and accidentally overwriting that space is bound to create very bizarre problems.
pjw has joined #riscv
jeancf has joined #riscv
Bigcheese has quit [Remote host closed the connection]
Bigcheese has joined #riscv
SwitchToFreenode has quit [Remote host closed the connection]
SwitchToFreenode has joined #riscv
geertu has quit [Quit: Changing server]
<pjw>
jrtc27: the u74 doesn't currently support ASIDs
Bigcheese_ has joined #riscv
Bigcheese has quit [Ping timeout: 272 seconds]
jeancf has quit [Ping timeout: 272 seconds]
<TwoNotes>
Well there is one less hting to worry about
<jrtc27>
pjw: thanks
<jrtc27>
one day hardware will :)
<jrtc27>
but given the TLB size in the Unmatched that's not hugely surprising
<sorear>
doesn't it have a fairly large L2 TLB?
<jrtc27>
oh you're right it does have a 512-entry TLB
<jrtc27>
*L2 TLB
<jrtc27>
though direct-mapped... :(
<jrtc27>
I think something I saw the other day only documented the L1 TLB
theruran has joined #riscv
valentin has quit [Remote host closed the connection]
ats has quit [Ping timeout: 272 seconds]
pjw has left #riscv [#riscv]
pjw has joined #riscv
jeancf has joined #riscv
<TwoNotes>
So, minimum size of an Sv39 page table is 3 pages, if all addresses kept under 2 MB?
<dh`>
minimum size is always 1, with large enough superpages :-)
jeancf has quit [Ping timeout: 272 seconds]
<dh`>
but it's 3 layers deep, so in some sense yes
mahmutov has joined #riscv
<TwoNotes>
Ok, so could stop at 1 or 2 if I use big chunks
<dh`>
yes, but in practice that's reasonably hard to arrange effectively
<dh`>
what are you doing? I forget
ats has joined #riscv
<dh`>
(or maybe you never said)
<sorear>
there's nothing actually stopping you from using the same page table for all levels
<sorear>
used to be called "fractal page tables"; works less well on riscv because leaf and non-leaf PTEs have different encodings, but still
<jrtc27>
I always knew it as recursive mapping
<jrtc27>
cute trick for memory-constrained systems
<TwoNotes>
dh` I am learning how microkernels work by writing one from scratch, eventually targeting BeagleV
<jrtc27>
irrelevant when you can map all of physical memory with a series of gigapages in one part of the virtual address space
<TwoNotes>
sorear, it says in my copy of the spec that the encoding is the same at all levers
<TwoNotes>
levels.
<sorear>
"same" always depends on a point of view
<jrtc27>
the format is the same, how you configure the bits is not
<TwoNotes>
Just those three mode bits
<jrtc27>
uhuh
<jrtc27>
and how do you distinguish leaf from non-leaf?
<TwoNotes>
Indexing algorithm is the same. PPN goes in the same place
<TwoNotes>
non-leaf has zeros in both R and X bits
<jrtc27>
so how are you going to reuse a non-leaf PTE as a leaf PTE?
<jrtc27>
you can't
<xentrac>
TwoNotes: awesome project!
<sorear>
the 386 paging algorithm always iterates twice; it's possible to use the same PTE at multiple levels because the PTE itself does not indicate leafness
<jrtc27>
(and when it introduced superpages, it did so with a superpage bit; x86 has superpage vs everything else, whereas riscv has non-leaf vs everything else)
<jrtc27>
(ie the difference is whether non-superpages are encoded the same as non-leaf pages or superpages)
<jrtc27>
though CALL vs CALL_PLT is a historical mistake in the RISC-V ABI, there's no need to distinguish them (and LLVM's linker, LLD, does not bother)
<leah2>
thanks!
<leah2>
i was on .72 still
iorem has joined #riscv
ats_ is now known as ats
cmuellner has joined #riscv
<dh`>
twonotes: I will suggest and strongly recommend that to begin with you put the whole vm system in your kernel, microkernel or not, as that will make it much easier to get things running.
<dh`>
you can always shift stuff to server processes later
<dh`>
(make them server threads within the kernel to begin with)