klange changed the topic of #osdev to: Operating System Development || Don't ask to ask---just ask! || For 3+ LoC, use a pastebin (for example https://gist.github.com/) || Stats + Old logs: http://osdev-logs.qzx.com New Logs: https://libera.irclog.whitequark.org/osdev || Visit https://wiki.osdev.org and https://forum.osdev.org || Books: https://wiki.osdev.org/Books
<zid> "?*
<tejr> choose life
<zid> choose a flat screen TV, with anime on it, about trains
<tejr> love it so far
<tejr> its shite bein autistic
<tejr> were the lowest of the low
<zid> I have an autismal scottish mate, he fucking loves a train
<zid> I send him train memes
<tejr> good friend
<epony> there are people for all kind of schools including railway transportation technical middle-high schools
<heat> serious q, is there a correlation between autism and love for public transportation?
<zid> yes
<Ermine> ah, gotta watch it on the last holiday day
<gog> tran nerd is worse but better
<tejr> lol
<tejr> you guys are cool
<gog> i am not cool
<zid> I am a transnerd, I surpass this reality's nerds
<heat> (X) Wrong
<epony> garbage truck simulator for the masses and their waste collector "labourers"
<Ermine> what if...
<Ermine> hmm...
<gog> oh i said tran
<gog> i meant train
<Ermine> or tram?
<gog> no
<epony> that's one of the lowest ratings of schools, only people with inability to classify in other technical schools end up there
<zid> trammy is derogatory
<gog> trams rights
<epony> railroad transportation is very trendy when public transport strikes hit the UK and France
<epony> like now
<tejr> trains is hard job
<zid> trains is hard job :(
<gog> i like strikes more than trains
<Ermine> our trams have screens on which chrome asks to execute xdg-open
<heat> gog, luv me a good strike
<zid> I like strikes more than most things
<epony> because.. retirement ages are pushed up towards what second world countries have, and the pay is not updates comparable to what political class get paid while jacking up inflation and spending of public resources on personal estates
<gog> luv me strikes, luv me direct acksion
<gog> 'ate austerity
<zid> 'ate scabs
<zid> simple as
<gog> simple as
<Ermine> we need more strikes
<zid> everybody needs more strikes
<heat> except baseball players
<heat> hehehehe
<Ermine> btw did UK govt react on recent strikes?
<epony> except unlucky strike smokers
<epony> who get fired from the job and get cancer at the same time
<zid> they reacted by going "lalalalalaa" I believe
<heat> uk government is now a react channel
<zid> "We'll give you 1% next year though? WHY ARE YOU UPSET?"
<epony> it's a servant to the monarchy
<heat> oi dis poor blokes are really shtupid innit
<zid> (inflation 8% and they haven't had a raise in 10)
<epony> feeding one family very well
<epony> well, who the fork wanted brexit..
<mrvn> But good that they quit the EU so the workers can't go and work somewhere with decent wages.
<geist> lets keep it on topic please
[itchyjunk] has quit [Ping timeout: 260 seconds]
<Ermine> sorry!
<gog> heh
<gog> yeah sorry
<zid> it's trains that are on topic here right?
<zid> but no strikes
<mrvn> is any chip factory on strike?
<immibis_> full-time OS developers on strike, all 12 of them
<epony> operating system development as a means to education and raising the skill in the developers learning about the system they are supposed to be programming
<Ermine> trains have operating systems
<immibis_> trains are operated with systems
<heat> them systems really do be operating
<epony> you can see railroad charts when you find out what EBNF is (and if you read a book about Pascal)
<heat> geist, did you see the 2GB jump thing?
<heat> pretty hilariously funny
<geist> hmm, no. was it a while back here? there's a bunch of nonsense noise
<heat> posted it to fuchsia #zircon too
<mrvn> Lastlog:
<mrvn> End of Lastlog
<mrvn> 01:12 < heat> geist, did you see the 2GB jump thing?
<geist> on discord?
<heat> yeah
<mrvn> can you post it again?
<epony> did you know that you can overcome the "process limit" barriers with overlay files and other sliding window techniques for working with larger files, so there is really no escuse for what Windows was causing to people for 10-20 years
<geist> oh heh yeah
[itchyjunk] has joined #osdev
<geist> i thought you could only relative jump 2GB anyway? or is it 4GB?
<zid> It makes sense that you probably shouldn't do that
<epony> "wingoes excuse"
<heat> geist, it's an indirect jump
<geist> oh i see
<mrvn> geist: I thought there is both
<heat> so jmp *(%rax) or something
<zid> we have tags and stuff for caching right
<zid> and they have so many way associativity and crap
<zid> makes sense that 'lots of other things' are limited by 2GB etc
<geist> yah i thought it was for relative jumps at first, which ithink you can only 2GB or so
<geist> but yeah for any sort of absolute or indirect. ouch.
<heat> probably heavily affects the zircon shotgun mappings right?
<epony> 2GB is the 32bit process size limit, 4GB is the (V)FAT(32) file size limit
<geist> yah but really if you look at how linux lays out a process, there's going to definitely be a jump between the binary and any lib
<heat> plus shared libraries on every system
fedorafan has joined #osdev
<heat> yep
<geist> yah in that case a shotgun layout isn't going to really be any different, unless some OS clusters everything together
<geist> and AFAIK they usually dont nowadays
<heat> it is tho?
<mrvn> geist: I wouldn't mind a memory model where .text of everything gets mapped into a 2GB block.
<heat> if you start jumping around libs and they're all > 2GB apart
<epony> there is one other factor, that is "machine generational" (rough estimate) of the memory that a machine can effectively process for 1-2 seconds as processor throughput and memory access/transfer rates
<geist> yah that's what i mean. i think most modern systems at least put libs probably >2GB apart
<heat> not individually, no
<geist> thoiugh i dunno maybe not
<epony> that's about those limits for the 32bit machines up to 2000-2004
<mrvn> geist: mmap usually takes from the end of memory but address space randomization changes that a bit
<heat> linux does: *put ET_EXEC elf at linked address, put ET_DYN PIE elf at ~0x555555555555*
<geist> yah i guess you're right. at least on this random linux x86-64 box, the binary is somewhere random, and then the 2 or 3 libs seem to be clustered somwhere else random
<heat> yeah then all libs are just straight up mmap'd, no magic
<heat> I was discussing on #musl about a patch to remove the 0x555555555555 thing, and hence remove the separation between the interpreter and the libs
gog has quit [Ping timeout: 252 seconds]
<heat> erm, sorry. remove the separation between the interpreter/libs and the PIE executable
<mrvn> heat: do you want to cluster them into a 2GB chunk or make all segments totaly random?
<heat> as things stand, PIE gets loaded to 0x555555...., interp gets loaded to 0x7fffffff...., and then mmap'd libs are placed under the interp
<heat> mrvn, wdym make segments random?
<mrvn> heat: roll a dice and put the interp there, roll again for .text, again for .data, again for each lib
<heat> you can't do that, shared objects have defined offsets between segments
foudfou_ has joined #osdev
foudfou has quit [Remote host closed the connection]
<heat> if .text 0, .data 0x200000, I addressof(.data) must be addressof(.text) + 0x200000
<mrvn> heat: not with PIE/PIC
epony has quit [Ping timeout: 268 seconds]
<heat> yes with PIE/PIC
<mrvn> then you didn't include enough relocation information in the elf.
<heat> the only thing that can get randomized is the load bias/load address
<heat> bud, there's no relocation information that makes this possible
<heat> you would never rip-relative address something
<heat> unless you start relocation patching .text. 1) that segfaults in current ld.so's 2) you broke the point of shared objects
<mrvn> heat: you patch the got
<heat> trivia: how do you access the got
<mrvn> heat: through the global register pointing to the got
<heat> must be a new register
<mrvn> got access is different on different archs
<mrvn> and flags
<heat> generally, you access the got through pc-relative addressing
<heat> geist, btw this whole thing sucks even harder because all calls into a shared lib go through the .got, so indirect call :V
<mrvn> heat: anyway, what is your goal?
<heat> idk, happiness, build a family, world peace
<mrvn> if you force libs to be mapped within 2GB of the PIE binary then you could relocate the .text and skip the .got
immibis_ has quit [Quit: HexChat]
<heat> you cannot because you're also writing on RO segments and killing COW
<mrvn> You do that at load time. And what COW? it's .text, it doesn't change post load.
<heat> load time?
<mrvn> in the dynamic linker
bauen1 has joined #osdev
<heat> what COW?
<heat> erm, 100 /bin/cat's share their whole text and rodata
<heat> because they are mapped COW
slidercrank has quit [Ping timeout: 252 seconds]
<mrvn> heat: if you don't want a trampoline then you have to sacrifice something.
<heat> but I do want it
<mrvn> You could have kernel support so all the 100 cat's use the same mapping. Go straight to the relocated .text on the next exec()
<mrvn> Or you could keep the trampoline but make it relative jumps instead of indirect
<heat> did you hear me say I don't want it? I want to exclusively pack shared libraries and the executable near each other as to make .got and this intel cpu fuckery work well
<heat> really, it's Intel that dropped the ball here
<mrvn> it sounded like you didn't want to do the indirect jump through the .got
<heat> I do want it, it's how shared objects work
Burgundy has quit [Ping timeout: 260 seconds]
<mrvn> if you have the indirect jump then why do you care if the binary and libs are close together? What am I missing?
<heat> the Intel CPU problem i posted where indirect jumps wider than 2GB mispredict a lot
<mrvn> 01:13 < mrvn> can you post it again?
<heat> sorry
<geist> well you have to more or less indirect it through got anyway because it could be far away
<geist> but if you did pack them tight you could avoid the got if you're okay patching the .text
<heat> yep, although the got thing really is to just save on dirty pages
<geist> yes and far away targets (on 64bit)
<mrvn> heat: so you DO want to cluster everything together so you can have relative jumps (32-bit signed displacement)
<heat> no
<heat> well, yes
<heat> I want jumps and calls to $random-64-bit-addr to be at most 2GB away so performance doesn't go down the drain
<mrvn> The wording on that page is pretty bad. It makes it sound like a "jmp *($rax)" will be better if the distance is small. But I think what they describe just depends on the opcode, not the actual distance.
<heat> and this is a much worse problem on non-PIE executables where you'll need to force shared libs down to < 4GB :(
<heat> mrvn, it will
<heat> that is the exact problem
<heat> there's no displacement in indirect jumps, it's all absolute
<mrvn> heat: You can read the text such that "JMP/CALL imm32" is more efficient.
<heat> well, no shit. those are not indirect
<geist> also by def they're within 2GB
<mrvn> or you can read it that any jump is more efficient when it could possibly be also written as "JMP/CALL imm32" instead of what it actually is.
<heat> no, you're reading it in random ways
<heat> it gives you an explicit example
<heat> wildly different misprediction ratios based on the distance to the PC
<mrvn> heat: nothing explicit in the example. Where is the opcode that they profile?
<heat> you should be less pedantic
<mrvn> heat: ant those branches are what? "jmp *($rax)"?
<heat> yes...
epony has joined #osdev
<mrvn> heat: The text doesn't convince me at all. For one thing the header talks about 21% misprediction but the tools output has no 21% in it at all. Secondly where is the same test with the library mapped close to show the predictor work better?
<geist> why dont you bring this up with intel and not heat, mrvn?
<geist> like, heat is just pointing out a thing, he didn't write it
<mrvn> don't know anyone at intel
<heat> i'm fairly sure that intel know their processor a lot better than you, such that this is not a random whitepaper nor errata but an actual section in the new optimization manual
foudfou_ has quit [Remote host closed the connection]
foudfou has joined #osdev
<mrvn> heat: they might very well be and probably are. They can be right without having shown proof.
<heat> for completeness sake... https://i.imgur.com/EfZgLj0.png
<mrvn> yep, that looks like a test of the same code with >2G jumps and <2G jumps.
<geist> it's an optimization manual, they dont have to *prove* it
<geist> they just say 'because of this thing you should avoid that'
<mrvn> They don't have to proof it. But they do give an "How to Detect it" that is inconsistent and doesn't show anything because it's only half the story.
<mrvn> or maybe there is another tool output on the next page, can't aay.
<mrvn> say
<epony> so.. how does the 2GB limit persist on 64bit systems, do you know?
<mjg> i'm pretty sure the test was only done on 64 bit
<mrvn> mjg: see the second url heat posted. You can just map things closer without having to go to a 32bit cpu.
<heat> it's interesting if this will spark any changes into operating system memory layouts
<heat> who knows if this will be present in future microarchitectures
<mrvn> How common is the "Golden Cove CPU"?
<heat> it's alderlake (desktop + laptop) and sapphire rapids (server, just launched a few months/weeks ago)
nyah has quit [Quit: leaving]
<epony> better use model numbers instead of code names
<epony> or CPU series / gen / year
<mrvn> I would really like to see the profile run they did redone with the 2G fix. How much of those 21% aka 22.30% misprediction at 0x555555603664 goes away when the distance gets smaller?
<epony> ah, the branch predictor artefacts ;-)
<epony> testing testing..
<epony> that's why game development for performance targets specific CPU models with particular chipset and periphery, also known as "consoles" which are just PCs with fixed specs and slightly different HW for "captive" reasons
<mrvn> Another thought: When the library function gets looked up for the got/plt the first time does it generate an "JMP/CALL imm32" when the offset is small enough?
<heat> no
<mrvn> that could be another optimization in the dynamic linker then.
<epony> imagine what happens when people reflash the firmware of their game console with a third party BIOS replacement with regards to that machine specific operation tuning
<epony> then figure out why are people doing that to themselves with alternate replacement for their system basic programming and expect some "dramatic" gains or achievements
dutch has quit [Quit: WeeChat 3.8]
<heat> geist, btw what does "wider machine" mean?
<mrvn> more phys bits? 5 level page table?
<geist> hmm, in what context?
<heat> the 1st screenshot
<geist> *probably* means it has more back end units
<geist> and/or decodes more instructions per cycle
<heat> "The Golden Cove CPU is a wider machine and might exhibit a higher Top-down Microarchitecture Analy-
<heat> sis (TMA) Bad Speculation percentage"
<geist> ah yeah, it's the 'lots of execution units, runs lots of instructions per cycle'
dutch has joined #osdev
<heat> ah and because of that it speculates further, and may mispredict more easily?
<Mutabah> I don't think its quite that bad
<Mutabah> Well.. it might mispredict more (as there's more in flight)
<geist> might be the mispredict penalty is higher
<Mutabah> but the pipeline depth is likely the same, so the penalty stays the same?
<Mutabah> (in absolute terms)
kanzure_ is now known as kanzure
<epony> typically width is used for the word, instruction, operand, data types etc and registers, address mode and data buses, so about the "bit width" of various parts of the computation and data transit
elderK has joined #osdev
heat has quit [Ping timeout: 246 seconds]
<zid> hmm my green has decided today it shall be stripey
<zid> too cold inside
<zid> one day the green will totally die and I'll have to try desperately to desolder some other device for matching caps to try :D
<mrvn> what's a green?
<geist> a lawn i guess? a golf thing too
<mrvn> indor? Maybe some virtual golf thingy that needs caps for the electronics.
<ornx> which EFI memory type is the good one? is it EFI_MEMORY_NV or EFI_MEMORY_RUNTIME
<Mutabah> Define "good"?
<Mutabah> Both of those are memory you shouldn't touch afaik
<ornx> conventional physically-backed memory
<ornx> wait those are attributes not types... EfiConventionalMemory is the right one i think
<Mutabah> at a guess from the names, the first is non-voltaile (flash or battery-backed RAM), the second is a scratch space used by EFI runtime servces
gxt has quit [Remote host closed the connection]
gxt has joined #osdev
<epony> sounds like a VGA cabling problem on an aging PSU in a monitor
<epony> and of course, cold starts make it happen faster, but summer heat is even more destructive so you experience that at winter (after drying up a bit more)
<epony> you have approx 1 month
<epony> to find a spare / replacement solution (or less)
<epony> peekaboo! one more reason to dislike the winter cold
<epony> you get the electronics failures just when you can't fix things
<epony> and that's nothing to do with "murphy's aphorism pseudo-laws"
gxt has quit [Ping timeout: 255 seconds]
gxt has joined #osdev
[itchyjunk] has quit [Remote host closed the connection]
dza has quit [Quit: ]
dza has joined #osdev
dza has quit [Client Quit]
dza has joined #osdev
fedorafan has quit [Quit: Textual IRC Client: www.textualapp.com]
dude12312414 has joined #osdev
srjek has quit [Ping timeout: 252 seconds]
smeso has quit [Quit: smeso]
Turn_Left has joined #osdev
Left_Turn has quit [Ping timeout: 256 seconds]
smeso has joined #osdev
bradd has joined #osdev
dude12312414 has quit [Quit: THE RAM IS TOO DAMN HIGH]
dutch has quit [Ping timeout: 248 seconds]
dutch has joined #osdev
aws has joined #osdev
les has quit [Quit: Adios]
les has joined #osdev
aws has quit [Quit: leaving]
aws has joined #osdev
bgs has joined #osdev
dutch has quit [Ping timeout: 248 seconds]
dutch has joined #osdev
ryzerth has joined #osdev
ryzerth has quit [Quit: Leaving]
ryzerth has joined #osdev
Burgundy has joined #osdev
slidercrank has joined #osdev
danilogondolfo has joined #osdev
gorgonical has quit [Remote host closed the connection]
gog has joined #osdev
fedorafan has joined #osdev
ryzerth has quit [Ping timeout: 248 seconds]
foudfou has quit [Remote host closed the connection]
foudfou has joined #osdev
nyah has joined #osdev
gmodena has joined #osdev
sinvet has quit [Ping timeout: 260 seconds]
x8dcc has joined #osdev
GeDaMo has joined #osdev
craigo has quit [Ping timeout: 246 seconds]
<x8dcc> hello, I added paging to my kernel as heat suggested but I am having some problems now. I filled the first entry of the table directory following the paging guide (https://wiki.osdev.org/Setting_Up_Paging)
<bslsk05> ​wiki.osdev.org: Setting Up Paging - OSDev Wiki
<x8dcc> before adding this, I was already using a simple heap allocation system directly on physical memory, and I think that's what broke after adding paging
<x8dcc> the heap had a fixed start address and size, but now I am not sure how to modify it with paging enabled :(
<Mutabah> Well, with paging on you now can lay out our memory however you like
<Mutabah> e.g. Put the kernel image at -2GB (assuming 64-bit) and the kernel heap at something nice and round, e.g. starting right at 0xffff8000_00000000
<sham1> I think the problem here is that the paging is being set up from inside the kernel, which is delicate to say the least
<sham1> It's easier to just set up the paging before going to the kernel proper and then just have your heap of pages
<x8dcc> my heap of pages? I don't understand that
<Mutabah> I think sham1 means "have your heap be built atop paging"
<x8dcc> yes, thats what I wanted to do
<x8dcc> thing is that although the heap is being initialized and all that *after* paging, its still not right
<x8dcc> pretty sure its because of the location of the heap
<x8dcc> I can send the code btw, but I'm pretty sure it's just that I don't get how paging really works :/
<sham1> Paging is weird to wrap your head around at firsr
<x8dcc> I think I understand the overall concept, but it's hard for me to figure out how to work with virtual memory and what not
foudfou has quit [Ping timeout: 255 seconds]
foudfou has joined #osdev
<x8dcc> for example, with what I have now (empty page dir except first table filled), could I initialize the heap in any address like I was doing before with physical memory?
<sham1> Yes, although then you also have to allocate a page for your heap
<x8dcc> well for example is hard for me to know when I have to allocate a new page
SGautam has joined #osdev
<immibis> when you want to use memory in a page you didn't already allocate
<x8dcc> immibis: what do you mean?
<Mutabah> Virtual memory is backed by physical memory
<Mutabah> A virtual page needs a physical _frame_ to be allocated and mapped to it to be used
<x8dcc> I see, thank you
<immibis> when you want to use address 0x12345678000 you have to map it to a physical page
<immibis> if you want to also use 0x12345678500 that's the same page so you don't have to change anything yet
<immibis> if you want to use 0x12345679000 you have to map it because it's a different page
<x8dcc> I think I understand. Sorry but this is new to me
<immibis> or you just guess a number of pages and map that many and when they're all used up you say the heap is full
wand_ has joined #osdev
Burgundy has left #osdev [#osdev]
wand has quit [Remote host closed the connection]
x8dcc has quit [Quit: leaving]
bradd has quit [Ping timeout: 248 seconds]
x8dcc has joined #osdev
weinholt has joined #osdev
randm has joined #osdev
SGautam has quit [Quit: Connection closed for inactivity]
craigo has joined #osdev
craigo has quit [Quit: Leaving]