<bslsk05>
lwn.net: Reconsidering the direct-map fragmentation problem [LWN.net]
<geist>
hmm, what triggers unmapping.. lets see
<geist>
on ARM thereis a very real problem that blocks of physical ram that are unavailable you need to unmap from the direct map or the cpu will prefetch into it
<geist>
we have this problem in zircon where we have to be very careful to only map precisely the pages that the kernel is responsible for
<geist>
x86 doesn't really have this issue
<heat>
x86 also has the issue
<geist>
okay
<heat>
with caching
<geist>
yah probably a more subtle version of it
<zid>
okay figured out the thing I was trying to figure out, yay?
<geist>
with ARM it's fatal
<heat>
there's a whole subsection in the SDM basically saying "there's a hack for the lower bits of the address space, but don't really do this mkay, kind of undefined?"
<zid>
I wanted to make my vidya game larger, it's an xbox port and renders at 480p. Making the window bigger -> big black border. Okay adjust the d3d viewport size -> still big black border. *gets stuck*
<zid>
It's not rendering with prims, it's just fucking bitblting the backbuffer.
<heat>
but I think the gist of it is that if you map a large page that has multiple MTRR ranges with differing caching, it gets all demoted to the more conservative one
<heat>
like a large page that has a UC and a WC will be effectively UC
<heat>
it's something like that
<zid>
I fixed, playable now, yay
<zid>
halved the backbuffer size wrt the backbuffer and now it scales the backbuffer to the viewport
<zid>
wrt the viewport*
<heat>
ok no sorry, it doesn't even necessarily demote the caching for the whole page, it's just a recommendation if you want to map e.g mmio and regular memory
<heat>
they have a hack that fragments large pages from 0 - 4MB into 4KB TLB entries
Left_Turn has joined #osdev
<geist>
heat: ah oay yeah the large page mtrr thing is true
<heat>
do you not unmap these bits from the direct map? I think I asked you a while back in the fuchsia discord
Turn_Left has quit [Ping timeout: 264 seconds]
<geist>
that's not strictly the same problem at all with the ARM one, but it's another direct map hazard
<geist>
oh i dunno, probably not doing the right thing on x86
Turn_Left has joined #osdev
<heat>
ah yes I see, your ARM problem is slightly different
<heat>
is that where you get an SError?
<geist>
yep
<heat>
keewwwwwwwl
<geist>
or worse you just get a system lock up
<geist>
an SError is when your SOC behaves
<geist>
basically if a mapping exists for a page and the cache bits are set on the page then the cpu is perfectly free to prefetch into it
<geist>
this is different from x86 where the caching/prefetchability/etc is generally determined by the physical range
<heat>
yeah I think x86 is just defined to return all 1s there
<heat>
or "defined" in practice
<geist>
ARM however will *never* prefetch into something marked device or uncached, so you are free to have mappings to that sort of stuff
<geist>
you can still accidentally touch it, but it'd have to be a stray pointer or a bug to trigger an Serror in that case
<heat>
i know x86 has the funny quirk of writes to ROM just possibly hanging the whole system
Left_Turn has quit [Ping timeout: 264 seconds]
<heat>
at least this was a big problem with the GDT :)
<geist>
it's all a manifestation of from the cpu's point of view the only metadata it can go on is via the bits in a page table
<heat>
riscv doesn't suffer from any of these issues does it?
<geist>
vs it being a two layer union of cache bits in the page table + physical properties like you get with x86
<geist>
well... riscv as it currently stands does not, but there are already extensions present to act more ARM like
<geist>
via vendors that probably are retargetting their existing socs with ARM cores
<heat>
Zaaaaaaaarch64
<geist>
caching on riscv is going to be a nightmare as more cores come out, because now you have the worst of both worls
<heat>
why?
<geist>
sifive cores (including upcoming ones) still hard stick to the rules that cachability is 100% determined by the physical address
<geist>
because going to end up with needing to dynamically deal with x86 style caching and ARM style caching
<heat>
ah okay
<heat>
yeah that sucks
<geist>
with various cache flushing instruction extensions, some of which are nonstandard
<geist>
and already seen various page table extensions to add more cache bits
<heat>
tbf x86 style caching is not that
<geist>
like the T-head cpus have their own version of it, and then there's a new extension that extends the page tables in a different way
<heat>
a true correct kernel tries to have the PAT bits correct as well, you can't just hang off of the MTRR
<heat>
in case the firmware people screw up (what???)
<geist>
yah but in general on x86 the phys8cal range overrides whatever you set
<geist>
*in general* i'm sure there are exceptions
<geist>
or more specifically the physical ranges can usually *lower* the cachability of a virtual page
<heat>
oh gosh yeah
<heat>
there's a whole table on that
<geist>
so you can usually map your pci apertures as cached and it'll just work, whereas that'd be totally fatal on ARM. you have to really have your ducks in a row there
<heat>
one day I want to see if I can coerce an x86 CPU into cache-as-ram
<heat>
should work in KVM, hopefully
<geist>
i do remember reading the family 15h bkdg it describes how to do it for initial firmware bringup
Left_Turn has joined #osdev
<heat>
the procedure is IIRC kind of described-ish around some places
<geist>
since the cpu starts running in XIP SPI flash, pre-dram controller brought up
<heat>
i'm not sure if the SDM has it
<geist>
it basically describes exactly how to set up the cache-as-ram situation so you can run more codes to set up memory
<heat>
yeah totes
<heat>
in intel platforms that's CLOSED SOURCE MAGIC TOP SECRIT
<heat>
(it ends up being really boring)
<geist>
yah though that's also 15h so i dunno how 17 and 19h does it
<geist>
though AMD announced they're moving off their closed source AEGSA so good for them
<geist>
something like OpenSIL? forget the name
<heat>
i think the semantics for all the CPUs should work pretty similarly, the CR0 caching bits and the MTRR probably work more or less the same
<heat>
yeah opensil
<heat>
it won't have memory training but you'll probably see a lot of PCIe link training and stuff
<heat>
fun.
<geist>
yeah really boring shit really. when you look at the amd and intel socs at this level they really dont look any different from an ARM soc
<geist>
just a bunch of soc specific stuff to set up to get it into a usable environment
Turn_Left has quit [Ping timeout: 240 seconds]
<heat>
yeah but CAR specifically doesn't really need platform magic AFAIK
<heat>
on x86
<geist>
bring up some sort of spi/i2c controller so you can talk to the flash on the dram sticks to read their parameters, then read from flash to get the bios params, then compute a bunch of dram crap, etc etc
Turn_Left has joined #osdev
* heat
nods
<heat>
i think they just wont open source it because of legality + patents
<heat>
like most of closed source x86 firmware, really
<heat>
i think opensil is a full rewrite from whatever was AGESA
<geist>
yah so in the case of AMD they probably just have tos tart over from scratch with opensil so they can avoid dealing with the crap in their current code
<geist>
yah
<geist>
as is usually the case it's not malice, it's the lawyers
<heat>
the intel DRAM training has like 20 something years of continuous existence
<heat>
lots of legacy, not trivial to rewrite
Left_Turn has quit [Ping timeout: 256 seconds]
<heat>
unless you want random breakage everywhere
<heat>
(AMD??)
<geist>
yeah only real solution there is to just start over with some new family, only support DDR6+ or something
<geist>
and then have two versions
scaleww has joined #osdev
heat_ has joined #osdev
heat has quit [Read error: Connection reset by peer]
GeDaMo has quit [Quit: That's it, you people have stood in my way long enough! I'm going to clown college!]
scaleww has quit [Quit: Leaving]
<heat_>
geist, yeah sadly I don't think their dedication to the open source!!!! is that large
heat_ is now known as heat
<heat>
as in they barely give a shit
<heat>
AMD seems to be somewhat negligent on the EFI side but they do invest in coreboot
<heat>
heck, amd doing a rewrite of their shtuff was pretty surprising to me too
innegatives has joined #osdev
<innegatives>
In x86 mul, CF and OF flags are set if "higher half of bits" contain any 1s. Does that mean higher 4 bits for byte sized instruction, and higher 8 bits for word sized insn?
<heat>
you should try reading more, it's a good idea
<heat>
as in "this is literally in the manual and I got the answer in 30 seconds"
<heat>
note: this is actual advice, not snark
<zid>
It's nice that you decided to make an instruction up btw
<innegatives>
what do you mean?
<heat>
mul is an instruction m8
<zid>
mul's an opcode, there's no mul instruction with an 8 bit result.
<heat>
oh yes
<zid>
ax = al * r/m8 is the smallest
<heat>
that's part of the "reading more" bit
<innegatives>
i read the manual, it doesnt say what high order bits are
<zid>
It literally does
<zid>
It even gives an exa
<zid>
mple
<zid>
*using mul*
<zid>
And I know that because I spent the last 10 seconds checking the manual
<innegatives>
Are you checking the instruction reference?
<innegatives>
its not there
<zid>
No, the entire manual
<zid>
the intro is where it defines terms
<zid>
what a byte is, what a word is, etc
<innegatives>
you mean volume 1?
<zid>
I used the combined 1/2/3 pdf
<innegatives>
there are 26 occurences of "high-order bits" in the combined volume, none of which explain what it is
<innegatives>
can you give me the page you are looking at please
<zid>
3.7.2
gog has quit [Ping timeout: 246 seconds]
<zid>
it's the first result for high order
<zid>
and asking fuz isn't going to help
<innegatives>
It says "Register pairs are represented with a colon separating them. For example, in the register pair EDX:EAX,
<innegatives>
EDX contains the high order bits and EAX contains the low order bits of a quadword operand.
<zid>
Correct
<innegatives>
So in case of AX, its just AH bits, in case of DX:AX, its just DX bits?
<zid>
yes
<innegatives>
ok thank you
<heat>
tldr read
<heat>
is there a particular reason why mul doesn't have a eax -> eax (etc) variant?
<heat>
while imul seems to
<heat>
tbf it's not like i can get the compiler to spit out a mul anyway, it's always imul
Left_Turn has joined #osdev
Turn_Left has quit [Ping timeout: 240 seconds]
<mrvn>
Huh? I thought EDX:EAX means just the 2 regiters concatenated.
<mrvn>
heat: other than flags mul and imul are the same.
<mrvn>
You need to do a mul and then test the result in a way where mul and imul differ and there is no equivalent comparison.
<mrvn>
heat: as to no eax -> eax variant... I think mul is intended for big integer multiplication where you want to multiply with eax over and over for a row.
thatcher has quit [Remote host closed the connection]
thatcher has joined #osdev
<zid>
heat why is mov [blah], imm32 longer than mov reg, imm32; mov [blah], reg :(
<zid>
(I know why, I just wanted to :( )
<heat>
:(
Gooberpatrol66 has joined #osdev
slidercrank has quit [Ping timeout: 265 seconds]
goliath has quit [Quit: SIGSEGV]
remexre has quit [Ping timeout: 268 seconds]
innegatives has quit [Ping timeout: 240 seconds]
<geist>
heat: yeah talked to some AMD engineers one time and they were super down on EFI
biblio has joined #osdev
dude12312414 has quit [Quit: THE RAM IS TOO DAMN HIGH]
<heat>
geist, alas, no alternative (yet)
<heat>
that does remind me that I should figure out the linux x86 boot protocol
<heat>
i've skimmed the docs before but it seems to me that half the thing is just deprecated fields
<heat>
may check out your linux boot stub thing
<geist>
yeah it ends up being like 5 fields you fill in from the current stuff
<geist>
and a bunch of padding
<heat>
well it probably depends on what version you're targetting no?
<heat>
i assume that doing an efi-exclusive thing is much easier than trying to support other older versions of the boot protocol
<heat>
ooh pkeys and shadow stacks are in the AMD manual now
gog has joined #osdev
Burgundy has quit [Ping timeout: 264 seconds]
Left_Turn has quit [Read error: Connection reset by peer]
rnicholl1 has joined #osdev
terminalpusher has joined #osdev
averymt has quit [Quit: Leaving]
rnicholl1 has quit [Quit: My laptop has gone to sleep.]
zxrom_ has joined #osdev
zxrom has quit [Ping timeout: 246 seconds]
zxrom_ is now known as zxrom
<heat>
geist, does/did darwin use a slab allocator?