klange changed the topic of #osdev to: Operating System Development || Don't ask to ask---just ask! || For 3+ LoC, use a pastebin (for example https://gist.github.com/) || Stats + Old logs: http://osdev-logs.qzx.com New Logs: https://libera.irclog.whitequark.org/osdev || Visit https://wiki.osdev.org and https://forum.osdev.org || Books: https://wiki.osdev.org/Books
heat has quit [Ping timeout: 252 seconds]
Burgundy has left #osdev [#osdev]
zaquest has quit [Remote host closed the connection]
zaquest has joined #osdev
Vercas68 has joined #osdev
Vercas6 has quit [Ping timeout: 258 seconds]
Vercas68 is now known as Vercas6
[_] has joined #osdev
[itchyjunk] has quit [Ping timeout: 272 seconds]
xenos1984 has quit [Read error: Connection reset by peer]
xenos1984 has joined #osdev
joe9 has joined #osdev
dennis95 has quit [Ping timeout: 260 seconds]
friedy has quit [Ping timeout: 268 seconds]
andreas303 has quit [Ping timeout: 268 seconds]
matrice64 has joined #osdev
k8yun has quit [Quit: Leaving]
Lumia has quit [Quit: ,-]
riverdc has quit [Quit: quitting]
riverdc has joined #osdev
[_] has quit [Remote host closed the connection]
matrice64 has quit [Quit: Textual IRC Client: www.textualapp.com]
elastic_dog is now known as Guest722
Guest722 has quit [Killed (zinc.libera.chat (Nickname regained by services))]
elastic_dog has joined #osdev
theruran has quit [Quit: Connection closed for inactivity]
Stary has quit [*.net *.split]
LambdaComplex has quit [*.net *.split]
Rubikoid has quit [*.net *.split]
zhiayang has quit [*.net *.split]
snickerbockers has quit [*.net *.split]
ebb has quit [*.net *.split]
kanzure has quit [*.net *.split]
Mutabah has quit [*.net *.split]
paulbarker has quit [*.net *.split]
brenns10 has quit [*.net *.split]
Rubikoid has joined #osdev
Mutabah has joined #osdev
kanzure has joined #osdev
paulbarker has joined #osdev
brenns10 has joined #osdev
snickerbockers has joined #osdev
zhiayang has joined #osdev
Stary has joined #osdev
LambdaComplex has joined #osdev
ebb has joined #osdev
axis9 has quit [K-Lined]
_whitelogger has joined #osdev
geist has joined #osdev
fluix has joined #osdev
Bitweasil has joined #osdev
graphitemaster has joined #osdev
sauce has joined #osdev
komodokitty has joined #osdev
colona has joined #osdev
pesdauskes has joined #osdev
komodokitty has left #osdev [#osdev]
cheapie has joined #osdev
pesdauskes has quit [Remote host closed the connection]
pesdauskes has joined #osdev
pesdauskes has quit [Remote host closed the connection]
ThinkT510 has quit [Quit: WeeChat 3.7]
ThinkT510 has joined #osdev
pesdauskes has joined #osdev
andreas303 has joined #osdev
dennis95 has joined #osdev
GeDaMo has joined #osdev
[itchyjunk] has joined #osdev
brenns10 has quit [Quit: The Lounge - https://thelounge.chat]
gog has quit [Ping timeout: 260 seconds]
poyking16 has joined #osdev
aejsmith has quit [Remote host closed the connection]
aejsmith has joined #osdev
[itchyjunk] has quit [Remote host closed the connection]
Burgundy has joined #osdev
vdamewood has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
wootehfoot has joined #osdev
poyking16 has quit [Quit: WeeChat 3.5]
gog has joined #osdev
sonny has joined #osdev
<sonny> Do you use inline assemly when needed or just have seperate assembly files?
<sortie> I generally prefer actual assembly files
<sortie> Inline assembly can be surprisingly tricky and dangerously error prone to get exactly right, so the compiler (or a future one) will not make any wrong optimizations
<sonny> I see
<sortie> If you must use inline assembly, generally get it reviewed by people in here and reuse fragments that have proven stable
<sortie> Whereas an assembly file is fully predictable (unless, of course, you make a mistake such as not following the calling conventions, e.g. for stack alignment and the other pitfalls, but at least it's deterministic)
<sonny> Thanks
<gog> yeah i have some wacky inline assembly and it's incredibly fragile
sonny has quit [Ping timeout: 244 seconds]
Matt|home has quit [Quit: Leaving]
sonny has joined #osdev
elastic_dog has quit [Killed (molybdenum.libera.chat (Nickname regained by services))]
elastic_dog has joined #osdev
vdamewood has joined #osdev
vinleod has joined #osdev
vdamewood has quit [Killed (platinum.libera.chat (Nickname regained by services))]
vinleod is now known as vdamewood
sonny has quit [Ping timeout: 244 seconds]
maxdev has joined #osdev
<maxdev> if I want to implement lazy loading for shared ELF libraries.. what's the best way of going about this? my plan was: load the pt_load section that contains the dynamic info; apply all relocations; for remaining load segments, store where they can be loaded from so they can be loaded on fault
<maxdev> is this reasonable or is there a more clever way
<gog> maxdev: the PLT
* vdamewood lazily loads a fishy into gog
<sortie> If you got a mmap mechanism, it'd be natural to mmap the binaries?
<gog> demand paging or lazy linking?
<gog> they're not totally unrelated concepts
wand has quit [Remote host closed the connection]
<maxdev> sortie i don't have that exact mechanism
<sortie> Might be an opportunity to build towards that
<maxdev> gog basically on-demand loading of the stuff in that binary to avoid loading all thats not needed
<gog> oh in the binary itself
<maxdev> sorry, the shared objects.
<maxdev> avoid loading stuff from the shared objects thats not needed
<maxdev> maybe also for the binary
<gog> ok think i understand
<gog> so the binary that depends on the library has a lookup function for unloaded symbols patched into it, and the library it depends on remains unloaded in memory until the lookup function is called by an unbound reference
<gog> ?
<sortie> I definitely want to properly implement mmap(2) demand loading on fault, so execve(2) can just mmap the program and such
<maxdev> gog: currently i relocate everything when loading the binary; so i thought about doing the fault-loading way
<maxdev> sortie: i'll need to read a little more about mmap
<sortie> It's worth studying :)
<gog> ok i understand, yeah you'll want to implement demand paging for that
<maxdev> sortie: i usually go the way of figuring something out myself, just to later see that someone else figured out something better lol
<gog> which if you already have a vmm and filesystem implementation it's not a big stretch
<gog> oh right
<sortie> maxdev, well you are asking how to fast load someone elses wheel :)
* gog bind chomp method
<maxdev> gog: yes that's not the issue actually - I just wondered, what I'll really have to load. and whether there is always a PT_LOAD segment containing the PT_DYNAMIC stuff.. because then it's rather easy
<maxdev> sortie haha :P
<gog> for DSO's, yes
<gog> there will always be a dynamic section
<maxdev> yes but will there always be a PT_LOAD segment containing this section?
<maxdev> i guess
<gog> yes
<gog> the ELF spec requires it because it's necessary for linking
<maxdev> okay then i'll load only the dynamic-section so i can relocate, and just keep info about the remaining load segments and load those on fault
<gog> demand paging can do that transparently
<gog> no need for extra bookkeepign about the image
<gog> and it's a more generalized type thing that can be used elsewhere instead of something specific for DSO's
<maxdev> wellll but i still need a little extra code for setting the empty space defined by p_memsz i guess?
<gog> yes you'll have to calculate that in advance but it's trivial
<maxdev> okay yes
<maxdev> i heavily refactored my elf loader so i can finally work on this, it bugs me a lot when executables load slowly :D
<gog> basically any page-aligned address > p_filesz is an anonymous page
<gog> just zero it out and map it in
* Mutabah spies ELF and groans
<maxdev> nice
<Mutabah> (Sorry, been making an ELF file all day - and I pop in and see more elf)
<maxdev> did you make it by hand in a hex editor?
<Mutabah> nah, with the `object` rust crate - but that just make the hexing easier
wand has joined #osdev
Vercas6 has quit [Remote host closed the connection]
<Mutabah> (hacking up something to convert a .sys into a .so... for reasons)
Vercas6 has joined #osdev
<maxdev> reasonable hackarino
<clever> Mutabah: sounds like a blog post i saw, where a guy converted a windows driver into a linux .so, then stubbed out every dynamic symbol it expected, so he could RE the driver under plain gdb
<gog> i've heard of obcopying elfs to be PE's but not hacking PE's to look like ELF's
<Mutabah> clever: Got a link, might save some time
<clever> *looks*
<Mutabah> My goal is quite similar, except slightly less stubbing
<clever> i had bookmarked it
<clever> 7 years ago....
<clever> guess my memory isnt failing, lol
<Mutabah> Neat. Not quite helpful (no relocation entries used), but neat
awita has joined #osdev
epony has quit [Quit: QUIT]
nyah has joined #osdev
maxdev has quit [Quit: Lost terminal]
eryjus has joined #osdev
eryjus has quit [Client Quit]
gmodena has joined #osdev
awita has quit [Ping timeout: 276 seconds]
vai has joined #osdev
<vai> Or have something more portable in mind people?
<vai> I need C + low level
<vai> physical memory addresses really needed to do a dynamic loader?
<vai> I am using only dynamic memory addresses in all my implementations
pesdauskes has quit [Remote host closed the connection]
<vai> okay, if the application or driver linked uses real memory addresses (physical addresses), like using implenting DMA access, probably in this case it would be good to date this up
<vai> but in this case I would rather call it up at system calls to get the physical memory address for relative one
Matt|home has joined #osdev
pesdauskes has joined #osdev
vai has quit [Remote host closed the connection]
Vercas67 has joined #osdev
Vercas6 has quit [Ping timeout: 258 seconds]
Vercas67 is now known as Vercas6
pesdauskes has quit [Remote host closed the connection]
pesdauskes has joined #osdev
dude12312414 has joined #osdev
Vercas6 has quit [Remote host closed the connection]
gxt has quit [Remote host closed the connection]
wand has quit [Remote host closed the connection]
gxt has joined #osdev
Vercas6 has joined #osdev
<gog> why is FileName in EFI_FILE_PROTOCOL.Open() not const-qualified
<gog> heat
<gog> where is heat
<gog> i need him to explain this baffling fucking design choice to me
wand has joined #osdev
wootehfoot has quit [Ping timeout: 272 seconds]
[itchyjunk] has joined #osdev
bslsk05[candyd] has joined #osdev
bslsk05[candyd] is now known as bslsk05
bslsk05 is now known as bslsk05[candyd]
bslsk05[candyd] has quit [Quit: :)]
bslsk05 has joined #osdev
bslsk05 is now known as bslsk05[candyd]
bslsk05[candyd] is now known as bslsk05
d5k has joined #osdev
xenos1984 has quit [Ping timeout: 252 seconds]
xenos1984 has joined #osdev
xenos1984 has quit [Ping timeout: 276 seconds]
Lumia has joined #osdev
xenos1984 has joined #osdev
nohit has joined #osdev
Lumia has quit [Quit: ,-]
xenos1984 has quit [Ping timeout: 276 seconds]
xenos1984 has joined #osdev
xenos1984 has quit [Ping timeout: 276 seconds]
xenos1984 has joined #osdev
sonny has joined #osdev
<sonny> So to get started with C for osdev you have to set up a stack right? I'm guessing this is the same thing with forth?
rorx has quit [Ping timeout: 260 seconds]
rorx has joined #osdev
heat has joined #osdev
<heat> gog, hello gog
<gog> heat
<heat> gog
<heat> why is it CHAR16 * and not CONST CHAR16 * you ask?
<heat> because there's usually no const correctness in C and if you look in the standard/UNIX land you'll find examples of it as well
<heat> but rest assured that argument is de facto const
<heat> as hinted by "IN CHAR16 *FileName"
<heat> if i had to take a shot every time i've seen string literals get passed to void f(char *) where the argument is never modified i would be in an alcoholic coma
<heat> sonny, yes, a stack would be more or less the only thing you need to set up when starting out
<sonny> ok ok
<sonny> things are starting to make sense now
<heat> idk about forth though
<GeDaMo> Forth traditionally has at least two stacks
<heat> 5th
<heat> why doesn't fourth have four stacks?????
<GeDaMo> It can! :P
<sonny> what would a process look like if it was made in forth?
<sonny> unix*
<sonny> what would a unix process look like
<GeDaMo> Save the old process state, load the new process state
<sonny> I mean the memory layout
<heat> how does the memory layout look right now and why is it not compatible with forth
<mrvn> does forth have memory?
<sonny> yeah, it has two stacks ...
<GeDaMo> The Forth "dictionary" is a linear address space
<GeDaMo> Just like a traditional heap
<heat> create a stack on the brk
<heat> easy
<heat> it's V1 unix compatible!
<sonny> ok, so it'd look the same then
<heat> nothing about a stack says that it should be growable
<heat> indeed, most stacks aren't, see pthreads
<sonny> stuff like RLIMIT_STACK?
<heat> RLIMIT_STACK can be viably set to whatever you initially mmap
<heat> and even if you have a growable stack, it may fail to expand due to $REASONS
<heat> like RSS/address space limits, another allocation right under the stack, etc
<heat> which actually reminds me, my VM system right now doesn't allow me to grow a mapping downwards
<heat> because of vm regions mapping to vm objects
<heat> if i want to grow downwards i'll need to heavily adjust the rb tree
<rb> pls no
<heat> if i had a radix tree i could maybe do something funky with the entries but im not sure
<heat> hehehehehehehe
<heat> and i thought my name was bad
<sonny> what was the single user system that was popular before unix?
pg12_ has quit [Quit: pg12_]
d5k has quit [Quit: leaving]
<mrvn> heat: do you actually need that? just make it really big and map pages on demand
<mrvn> or expand it by allocating 2^N pages every time it runs out incrementing N every time. Only causes O(log n) tree entries then
<mrvn> sonny: What was the popular horse breed before the Ford Model T? You are comparing apples and oranges.
<sonny> whatever point I had has been forgotten
<mrvn> I'm not sure single user systems had an OS at the time unix came to be.
GreaseMonkey has joined #osdev
<bslsk05> ​'Say Hello to my New Wangwriter 5503!' by Usagi Electric (00:28:25)
<clever> a z80 computer/printer, that i think originally only booted directly into word-processing software (he is currently missing the OS disk)
<clever> but it later had CP/M ported to it
<zid> wangwriter? nice
<zid> rabbit electric makes them sound like a dildo company too
<clever> that wang was purchased in 81
<clever> but wikipedia says unix was released in 69, dang, we need to go older!
<geist> hello fronds
GeDaMo has quit [Quit: Physics -> Chemistry -> Biology -> Intelligence -> ???]
<gog> there should be const correctness
<geist> yah i've started to const local vars and arguments to functions too. still works in C just fine, just something i have been picking up as a good practice from work
<mrvn> gog: correct
<geist> especially those complicated functions where you may have loops that modify a lot of state
* gog pets geist
* geist purrs
<geist> also it's raining! for like the firt time in forever. a very delayed fall here
<mrvn> geist: sometimes I want to mark variables as being const for a block in a function.
sonny has quit [Quit: Client closed]
<mrvn> but then that should probably just use helper functions
<geist> yeah
<geist> feels like a rustism bleeding into C/C++. though i'm sure it's a practice in C++ a long time, i'm only starting to see widespread use of it fairly recently
<mrvn> const correctness?
<mrvn> or helper functions?
<geist> const local variables and args
<geist> not pointers to const things, that's a good idea for a long time
<geist> but like
<zid> definitely read that as racism
<geist> int foo(const int bar, const char * const baz) etc
<mrvn> const args is pretty controversial for value types
<geist> really? why would it be? it's just a local variable that happens to be auto assigned by the compiler
<mrvn> You could say that "const int bar" is a good reminder that changes to "bar" won't survive the end of the function.
<mrvn> geist: it's a copy, why should you prevent modification. It won't affect the caller.
<mrvn> int foo(const int bar, const char * const baz) { int var_bar = bar; ... } is a bit silly.
<geist> sure, think of it this way. if you have a local variable that you set and dont modify, marking it const is a marker that tells the programmer as such
<geist> and then if you extend that to args, they're just local variables at function scope
<geist> what i'm not saying is you *have* to do it, more like, if you dont locally modify it, mark it const, otherwise dont
<mrvn> yeah, local vars make them all const till you get an error that you modify them. Good practice.
<geist> right, so i'm saying that since args are also local vars you can extend it to that
<geist> it doesn't modify the function sig either, so you only hae to do it locally
<mrvn> sure. I'm just mentioning that it's controversial. No value judgement.
<geist> gotcha. yeah i'm seeing it more at work, and like i said i suspect it's a bleed over from rust, where everything is basically const unless you mark it mut
<mrvn> Anything not a value type should be const for sure. But it's hard to always know what is a value type in C++. For fundamental types most people didn't use const for the longest time.
<geist> as a general rule i try to avoid modifying argument variables anyway, unless it makes the code substantially worse as a result
<geist> no particular reason, as i said arg vars are really just locals at the function scope, but i think psychologically as you're looking at the function you assume the value is the same across the entire func
<mrvn> My argument is always: If you mark it const and modify it by accident the compiler will tell you, catching your error. If you don't mark it const and forget to modify it ... well, that never happens.
<mrvn> wouldn't have an effect outside the scope if you modify it anyway
<geist> right
<mrvn> But it's more a "do as I say, not as I do". Have to get more into that style myself.
<geist> as a side note i think that's basically why the old style K&R argument declaration is the way it is. logically it's saying 'match the args listed here with this list of local variables, declared in the scope just 'outside' the function body
<geist> there's essentially an implicit scope after the function name but before the function starts that the arg vars live in, basically
<mrvn> think of it this way: <function + number of args> <stack frame layout> <code>
<mrvn> remember that all variables have to be declared at the start
<geist> oh sure, i'm just saying that's probably what they were thinking about with the K&R style arg stuff, which modern folks find completely baffling
<mrvn> and args are passed on the stack
<mrvn> *nod*
<geist> <function><implciit scope><arg variables><function scope><local variables><code>....
<geist> and yeah passed on the stack also kinda makes sense, yeah
<mrvn> parse function name, parse args ==> convert to stack frame offsets, parse local vars ==> convert to stack frame offsets, code ==> replace idents with stack references
<mrvn> So the code gen just gets stack offsets to work with and doesn't have to deal with any variable names or anything.
<geist> yah that's true too, as early C compilers were it'd also be much simpler to parse it
<geist> since you're now parsing things one step at a time
<geist> instead of the somewhat more complicated parse of a mixed function + arg decl in one line
<geist> anyway i just spent last night hours cleaning up the x86 32 and 64bit mmu code in LK. now i think it's in okay shape, though the flow of the code is basically the same. it's at least got a lot more type safety and whatnot
<geist> or at least as much as plain C lets you get
<mrvn> geist: isn't it so you can declare a function without types and then have the function definition look the same?
<geist> well that's true too
<geist> which stricktly speaking isn't any worse than writing things in asm the way they had been doing
<geist> ie, there's a function called foo. you had better know how to pass args to it, because nothing will enforce it
<mrvn> for me, with a modern parser generator, the old style actually seems much harder.
<geist> yah probably
<mrvn> I worry about such things like declaring the function with 3 args but then only defining 2 or defining 4.
<geist> i'm always reminded that there were C compilers in the early days, or on 8 bit micros that fit in tens of K of code, or less
<mrvn> foo(bar, baz) int bar { }
<geist> there's even a PDP8 compiler that fits in 4k words of code (6KiB since 12 bit words)
<mrvn> tens of K? Are you insane? That leaves no ram for the code.
<geist> and it's pretty amazing it works
<geist> mrvn: why is that a problem? you write it out to disk and read it back in again
<mrvn> true. if you do multi pass you need that anyway
<geist> compiler reads in C one line at a time, since it's single pass anyway by definition (also an artifact of it being wrtten on old machines)
<mrvn> But I think they had single pass compilers too
<geist> blat out the .o and then the assember gets the problem of probably being multipass
<mrvn> read source stream and output binary stream directly.
<geist> but lots of assemblers were in the day
pg12 has joined #osdev
<geist> exaclty, that's my point, a C compiler that outputs asm basically already is single pass because that's what you did on old machines where you couldn't hold the compiler and all of the source and object in memory at the same time
<geist> there's a fairly great compiler for z80 CP/M machines called BDS that runs in probably 32K code, but is actually pretty good
<geist> ie, isn't just replacing C lines with function calls to a lib, like lots of cheapo compilers do for 8 bit machines
<clever> there was a bootstrapper example i saw years ago called bcompiler
<clever> and the final C language, was more of a RPN based language that was entirely macro based
<clever> at the root level, the macro's just barf out raw assembly as binary data
<clever> and everything else is just macro's chained together to make that usable
<clever> the first stage was more of just a dumb hex->binary program, followed by hex->binary bit with labels, so you didnt have to compute byte offsets
<geist> yah and i've seen lots of 8 bit compilers that basically just do a first pass constant folding/etc on your input language and then just direct convert the results to either trivial inline code or calls to external library routines
<geist> and then just statically link that in
<geist> microsoft's BASCOM on CP/M was similar. compiled basic to native code. ran a lot faster, but the codegen was pretty dumb
<geist> for the most part most of the alu ops were library calls since basic was like 32bit floating point everywhere
<geist> but even that trivial stuff was still quite a bit quicker than an interpreter of basic
<clever> i lost the link, but i think i saw it on https://www.youtube.com/c/adriansdigitalbasement/videos ?
<bslsk05> ​'Adrian's Digital Basement - Videos' - 'Welcome to Adrian's Digital Basement. I love old computers! If you do too, this channel is for you! This channel is just a result of my hobby, so please be kind when my production quality doesn't stand up to those of other full time content producers.
<clever> where he was showing a fortran? compiler on a c64
<geist> yeah probably
<clever> and it basically just generated raw asm every time you input even 1 line of code
<clever> and then it just needs to keep a symbol table
<clever> and one weird thing, is that while you could modify a function, it wouldnt update all references to it
<clever> it basically just overwrote the entry in the symbol table, but the calls are by absolute addr
Vercas6 has quit [Quit: Ping timeout (120 seconds)]
MuonNeutrino has joined #osdev
<geist> oh nteresting, quickbasic 1.0 was just a rebranded BASCOM for DOS
<bslsk05> ​'Programming in FORTH on Commodore 64' by 8-Bit Show And Tell (00:32:59)
<clever> aha, it was forth, not fortran
Vercas6 has joined #osdev
<geist> yah that makes sense. forth you can run on almost nothing
<clever> that looks like RPN and also reminds me of lua's internals
<CompanionCube> clever: https://github.com/certik/bcompiler
<bslsk05> ​certik/bcompiler - Mirror of http://www.rano.org/bcompiler.tar.gz, with a bootstrap script (16 forks/74 stargazers)
<clever> CompanionCube: yep, thats the one
dude12312414 has quit [Quit: THE RAM IS TOO DAMN HIGH]
<bslsk05> ​github.com: bcompiler/header.bc at master · certik/bcompiler · GitHub
<clever> it doesnt even really understand elf
<clever> its just a set of raw bytes it emits at the start of any output
<mrvn> re
<mrvn> i, j, k, l, m, n are integer unless decalred float :)
<mrvn> I remember C64 Basic. String variables used $. I can't remember the syntax for int/float but I'm sure it didn't use float all the time.
<mrvn> And unlike QBasic on PCs the C64 tokenized the basic program on input and run that. The saved source code was the tokenized data.
heat_ has joined #osdev
heat has quit [Ping timeout: 276 seconds]
<heat_> <mrvn> heat: do you actually need that? just make it really big and map pages on demand
<heat_> yes, I do
<heat_> big mappings are incompatible with no-overcommit
<heat_> it also doesn't cover every case for growing downwards
<mrvn> true. but marking the stack VM object as overcommit might be easier than rewriting your tree
<heat_> imagine an anon mapping at [0x2000, 0x3000]
<mrvn> nobody needs more than 1TB stack :)
<heat_> someone maps another anon region at [0x1000, 0x2000]
<heat_> you should merge these two
<heat_> Real VM Systems (like linux et al) will merge everything they can
<mrvn> oh, if you can merge regions than growing the stack is trivial. just allocate the page before it and merge.
<sortie> heat_, actually, SHOULD they be merged??
<mrvn> My kernel doesn't even allow that. all allocation have a blank page between them.
<heat_> if you have [0x1000, 0x2000] and [0x4000, 0x5000], and then you map [0x2000, 0x4000], it gets merged into [0x1000, 0x5000]
<heat_> it's the optimal solution
<sortie> heat_, that means munmap(2) might now fail on a region you previously allocated because the kernel could OOM when growing the mapping table again in case you poke a hole in a region
<heat_> yes, this is all about merging and growing
awita has joined #osdev
<heat_> sortie, yes, they should be merged. maybe not an rfc SHOULD ;)
heat_ is now known as heat
<sortie> E.g. whether you want to provide the property that you can always munmap something you mmap'd
<mrvn> sortie: only if it is unpopulated
<heat> if the kernel OOMs when growing a table, you either panic or get some stable, consistent state back
<sortie> Panic on OOM?
<sortie> You must be Linux
<heat> yes
<heat> OOMs are like, the last resort :)
<sortie> Whatever happened to NULL and ENOMEM
<mrvn> OOM failure on freeing memory is pretty ironic
<heat> munmap will always be required to possibly allocate memory in traditional vm designs
<mrvn> On success, mmap() returns a pointer to the mapped area. On error, the value MAP_FAILED (that is, (void *) -1) is returned, and errno is set to indicate the cause of the error.
<heat> if I split [0x1000, 0x4000] i'll need a new struct vm_region
<heat> oopsie!
<mrvn> munmap may fail, it's designed for it.
<heat> yeah
<sortie> A munmap that can fail, very cute toy os
<heat> but point being that the only time you'll see an OOM here is probably if the kernel is literally dieing
<sortie> idk OOMs are pretty natural
<heat> you already swapped everything you could, flushed everything you could, your memory caches have gone to shit, your slabs are reduced to the minimum
<sortie> Doesn't mean things are dying
<mrvn> sortie has a point but heat too.
<sortie> Yeah I'm being all stubborn here but actually I literally can't quite make up my mind
<heat> if your system fails to allocate 100 bytes you're pretty much useless
awita has quit [Client Quit]
<sortie> heat, why? You just fail to do that, and the program trying to do something fails, presents a message, etc.
epony has joined #osdev
<heat> your system stops being able to operate correctly
<heat> and what's an operating system without operation? :(
<mrvn> sortie: I think heat agrees with you there. munmap simply returns -1
<sortie> Sounds like you overcommitted and didn't allocate resource limits appropriately
<heat> no, nothing to do with overcommit
<heat> this is a problem on non-overcommit too
<sortie> See the second part of the sentence :P
<sortie> But yeah agreed
<heat> munmap should return -ENOMEM here, or panic
<heat> either one is fine IMO
<heat> i don't remember what linux does. I think it's able to return -ENOMEM here
<sortie> Panic sounds like a local denial of service vulnerability
<heat> spoiler: most systems are vulnerable to OOM DoS :P
<mrvn> I feel that panic is defiently wrong. OOM kill the process would be borderline OKish
<heat> if you have no memory to allocate in general and return -ENOMEM, you DOS'd everyone too
<heat> you stop being able to do like, most of anything really
<sortie> I mean that's what heat suggests, heat will only OOM in the kernel after all avenues of clearing up memory has failed
<sortie> Although OOM killing processes is not my style
<heat> yes, that's how most kernels work
<heat> traditional kernels use as much memory as they can until shit gets tight
<heat> then you cut back on memory and see if you can stay above the watermark
<mrvn> sortie: If you don't OOM kill then basically every process will turn into an ENOMEM loop when you reach resource limits.
<sortie> Not that Sortix has its OOM shit together. I think I stopped panicing as much as I used to due to bad compiler optimizations
<heat> that reminds me, i have that important -fcheck-new patch for llvm lol
<heat> i should re-ping everyone
<mrvn> what does that do?
<heat> fcheck-new?
<heat> removes the implicit [[nonnull]] on the return value for void *operator new(size_t)
<mrvn> always or on noexcept?
<heat> llvm just eats it and pretends it does what it says. it doesn't.....
<heat> always
<heat> it's popular in noexcept code
<mrvn> noexcept new can return nullptr so that must be checked
<heat> -fno-exceptions doesn't remove the nonnull unfortunately
<heat> maybe noexcept yeah
<heat> but that's super verbose IMO
<heat> i hate it
<mrvn> maybe -fno-exceptions should remove the non-noexcept new :)
<heat> gcc -fcheck-new removes the nonnull so your nullptr checks don't get optimized out
<heat> clang -fcheck-new eats the option and pretends it understands it, but optimizes out your nullptr checks
<mrvn> What I'm missing is a way for the constructor to fail in no-except new.
<heat> which is... nonobvious
<heat> and potentially super dangerous
<sortie> ^^
<sortie> kernel/op-new.cpp-#ifdef __clang__
<sortie> kernel/op-new.cpp:#warning "security: -fcheck-new might not work on clang"
<sortie> kernel/op-new.cpp-#endif
<sortie> I listened to heat
<mrvn> heat: oh, optimizing null checks out, evil. I had code that called a function with a nullptr reference so I added "if (foo == nullptr) PANIC();" and it didn't panic.
<heat> heh
<mrvn> s/foo/&foo/
pesdauskes has quit [Remote host closed the connection]
<heat> well, that's UB :D
<mrvn> Nothing UB there. It's perfectly define to be always false
<heat> throwing operator new is defined by the spec as "never returns null", so llvm tags the return value as nonnull, and does HAXXOR HAX and removes your checks
<heat> yes it is
<heat> a nullptr reference is UB i'm pretty sure
<mrvn> heat: the caller is totaly UB. The above code is not.
<heat> which makes it so &foo == nullptr is false always
<heat> so optimized out
<heat> checkmate mrvn
<heat> c++ standards committee 1 - 0 mrvn
<mrvn> same problem you have with "new". the return value is non-null so your check is always flase and optimized out
<heat> yeah
<mrvn> Problem is: How do you check that in code and PANIC?
<heat> your specific issue?
<mrvn> I know the code is not supposed to call the function with nullptr. I also know the code does call it. I want a PANIC so I get a stack backtrace to find where it comes from.-
<heat> maybe a __asm__ __volatile__("":::"memory")? or something similar
* sortie . o O (Chess with undefined behavior, where if your opponent makes the wrong move, you can just immediately move your piece across the board, batman appears to slap the opponent in the face, and you descend through the floor into the abyss)
<mrvn> heat: it's a reference. It's still non-null after clobbering
<heat> ubsan?
<heat> ubsan will save your ass in that situation
<mrvn> something I have on my ToDo list for the kernel.
<heat> there's also a funny flag -fsanitize-trap
<heat> which does ubsan checks but traps instead of calling ubsan library code
<mrvn> You can convert the ref into a volatile pointer and then nullptr check that.
<bslsk05> ​godbolt.org: Compiler Explorer
<mrvn> YOu do need volatile or clobbering or something to break the compiler optimizer to remembering the non-null flag.
<mrvn> heat: ??? what's that supposed to be?
<heat> ubsan doesn't care about nonnull
<heat> it's fsanitize-trap
<heat> it does the UB checks but ud1's
<heat> fairly sure it works for int&
<mrvn> ahh, didn't see the compiler flags
<heat> ah, it doesn't
<heat> fuck!
<heat> wait
<heat> it does!
<bslsk05> ​godbolt.org: Compiler Explorer
pesdauskes has joined #osdev
<heat> yeah it doesn't
<heat> shit
<heat> it'll probably assert at the callsite though!
<bslsk05> ​godbolt.org: Compiler Explorer
<heat> yass queen
<bslsk05> ​godbolt.org: Compiler Explorer
<mrvn> heat: the caller is UB; the function is not. So the caller gets the ud1l opcode
<heat> yup
<mrvn> hehe, same idea
<mrvn> heat: you should enable "demangle identifiers"
<heat> boooo
<heat> i love muh mangled
<heat> yeah same idea but mine shows trap-ubsan in full effect
<heat> it checks for null and alignment in mine
pesdauskes has quit [Remote host closed the connection]
<heat> yours just optimizes it out to ud1, which it would do even if you didn't have fsanitize-trap
<heat> also, just found out unfortunately gcc doesn't have this mode I think :(
<mrvn> without fsanitize-trap you get: fsanitize-trap
<mrvn> callq __ubsan_handle_type_mismatch_v1@PLT
<heat> yup
<heat> which requires the support code you don't have
<mrvn> I should look into the fsanitize-trap. That should be trivial to implement in the kernel.
<heat> btw there's a minimal ubsan mode in llvm compiler-rt, you may want to look into that
<heat> ubsan itself is also not that hard
gxt has quit [Remote host closed the connection]
<mrvn> No need to implement any sanitizer hooks, just the opcode exception
<bslsk05> ​clang.llvm.org: UndefinedBehaviorSanitizer — Clang 16.0.0git documentation
<heat> i originally stole sortie's ubsan
<heat> it's super portable and fine, but it's a bit too limited
<heat> i've been meaning to replace it
<mrvn> Propblem is I use gcc. :( And I don't think it has that mode.
<sortie> thief
gxt has joined #osdev
<heat> sortie, should've kept it GPLv3
<sortie> heat -- wait
<sortie> heat, clang copied my code??
<heat> waitpid()
<heat> no
pesdauskes has joined #osdev
Vercas6 has quit [Ping timeout: 258 seconds]
Vercas6 has joined #osdev
tsandstr has joined #osdev
<tsandstr> I am starting to write my first toy kernel, and I am running into some trouble. I am trying to use Multiboot2 with GRUB under UEFI on x86_64 (although I am not trying to use EFI boot services). My multiboot header only contains tags with an entry address and requesting that my kernel is aligned. So far as I can tell from the Multiboot2 spec, it seems like when entering my code, the CPU should be in 32 bit protected
<tsandstr> mode. However, GRUB is handing me a 64 bit pointer to the EFI system table. How can I use this pointer? Do I need to enter long mode before I can gain access to the EFI system table?
<mrvn> Here is how you optimize your code: https://www.youtube.com/watch?v=c33AZBnRHks
<bslsk05> ​'Someone improved my code by 40,832,277,770%' by Stand-up Maths (00:28:46)
<mrvn> tsandstr: why would you need the EFI system table?
<mrvn> You probably need PAE for 32bit mode if you want to use the EFI system table at such an address
<heat> you need 64-bit yeah
<heat> i had the impression they dropped you off at 64-bit if you didn't exit boot services
<heat> huh
<mrvn> heat: not with efi -> grub -> kernel
<tsandstr> mrvn: Before I get too deep into this, I just wanted to have an eay way to print to the console. A simple 'Hello World' would help keep me motivated, and from what I've read the EFI system table is the easiest way to print things out
<heat> yeah sure, so jump to 64-bit
<heat> glhf
<mrvn> tsandstr: but then you are trying to use efi boot services
<mrvn> (or non boot services)
<heat> tsandstr is still in boot services
<tsandstr> I am do I need boot services to print? I thought I could disable the boot services but still use other services
<heat> wait.
<CompanionCube> alternatively you could see if the CSM's VGA/VBE emulation was started
<heat> yes, you do
<mrvn> tsandstr: the easiest is to print on the serial port or the VGA text mode.
<heat> printing is a boot service
<heat> printing to serial is the easiest
<heat> in an emulator you can just outb a few bytes to COM0's out port
<heat> no need for extra code
<tsandstr> heat: Okay, sounds good. I'll try that out. When I am reading the UEFI spec, how do I tell what requires boot services and what doesnt? Somehow I was under the impression that printing wouldnt use boot services
<heat> everything under EFI_BOOT_SERVICES requires boot services lol
<CompanionCube> aren't the non-boot services usually also explicitly called 'runtime services'
<heat> ah wait, output isn't under EFI_BOOT_SERVICES
<tsandstr> I was only using SystemTable->ConOut->OutputString, so I thought I was avoiding anything in bootservices
<heat> still, yeah you need BS
<tsandstr> heat: ha, okay, that's why I was confused lol
* heat rtfms
* pesdauskes executing self-destruct on enemy unit ...
<heat> yeah I think EFI_RUNTIME_SERVICES is the only stuff that's considered "runtime services"
<heat> aha, found the legalese
<heat> "After an operating system has taken control of the platform with a call to ExitBootServices() , only the Hdr , FirmwareVendor , FirmwareRevision , RuntimeServices , NumberOfTableEntries , and ConfigurationTable fields are valid."
<tsandstr> Aha! There it is
<tsandstr> Thanks for helpin me find that
<pesdauskes> all apic is reprogrammable, you can get thousands of records about different "hardware" j stalled
aleamb has joined #osdev
* pesdauskes waits for amd bios update with unlocked cores
* pesdauskes runs away after pausing the dialog
<mrvn> "This simple heat engine has no moving parts, and the top will spin as long as there is heat from the candle. " It doesn't move but spins, ok. Spinning is not moving, right.
elastic_dog has quit [Ping timeout: 272 seconds]
elastic_dog has joined #osdev
heat has quit [Remote host closed the connection]
tsandstr has quit [Ping timeout: 272 seconds]
heat has joined #osdev