klange changed the topic of #osdev to: Operating System Development || Don't ask to ask---just ask! || For 3+ LoC, use a pastebin (for example https://gist.github.com/) || Stats + Old logs: http://osdev-logs.qzx.com New Logs: https://libera.irclog.whitequark.org/osdev || Visit https://wiki.osdev.org and https://forum.osdev.org || Books: https://wiki.osdev.org/Books
gog has quit [Ping timeout: 260 seconds]
* kof123 quietly puts on shades to avoid blink of death
<heat> gog
Vercas has quit [Quit: Ping timeout (120 seconds)]
Vercas has joined #osdev
gxt__ has quit [Remote host closed the connection]
gxt__ has joined #osdev
troseman has joined #osdev
wand has quit [Remote host closed the connection]
<heat> damn left me on read huh
wand has joined #osdev
<zid> left you on unread
<heat> gog's favourite dark souls boss is capra demon
troseman has quit [Quit: troseman]
* moon-child tries to pet gog, but hands pass harmlessly through the air
* sakasama emits ionising radiation.
* moon-child pets sakasama
zaquest has joined #osdev
* sakasama 目からビーム's moon-child.
<bnchs> mew mew mew
<bnchs> heat: there's nothing wrong than the entire channel going silent as you say something utterly retarded
<bnchs> nothing worse*
NicePotato has joined #osdev
slidercrank has quit [Ping timeout: 260 seconds]
<geist> heat: word. re tianocore riscv
<geist> who was doing the work? I wouldn't be surprised if some of the new server riscv startups are starting to get serious about it
<geist> ie, aint got not time for no uboot on a server
<heat> ventana micro
<heat> i think they've been doing good work all over the low level stack on riscv
<heat> now Intel is in on it, so was HP
<geist> ah yeah ventana. yeah they're one of the ones that are going to try to make Big RiscV
<heat> "EDK2 for qemu virt is booted in S-mode as a payload for M-mode FW"
<NicePotato> I am struggling to figure out where to start when implementing a GDT and a IDT I have followed the meaty skeleton thing, and added to it (currently all of the sample inline assembly functions and pcspeaker) I have looked at the GDT and GDT tutorial pages on the wiki, but I am not understanding it. I have looked at others implementations, but I
<NicePotato> still have no idea how they work. Any easier to understand (not simpler) tutorials?
<NicePotato> I have a github of my source if thats wanted
<heat> what part do you not understand?
<geist> also it will sound a bit like a broken record, but the intel and AMD manuals do describe what the IDT and GDT are for and what the bits mean
<geist> not that they're necessarily any clearer
<geist> but you should be able to cross reference what the wiki and whatnot are against the manual
<NicePotato> I don't know where to start implementing it into my kernel. I understand what they do, but not how to implement them into my operating system
<Mutabah> The GDT should just be a hard-code and forget thing really
<NicePotato> Am I supposed to use asm files, or add them to some of those .S files?
<geist> well, in the abstract you need to have at least a starting GDT, can have it precomputed, and load it once atkernel start
<geist> ah. well you can do it any number of ways, including defining and loading them in C
<Mutabah> `.S` is the extension for the gnu assembler
<geist> the only piece of ASM tou actually need is the lgdt and lidt instruction
<Mutabah> `.asm` is traditionally nasm/fasm (anything that uses intel syntax natively)
<NicePotato> I used NASM before when I tried creating my system in real mode
<geist> think of it as just a in memory data strcutre that has to be 'loaded' into the cpu
<heat> if you understand what they do, what part do you not understand?
<geist> ie, you create this table somewhere in memory (hard coded is fine) and you use the lgdt instruction to point the cpu at it
<geist> you can do that in either asm or C. lots of examples will use asm, which is fine
<heat> whether they are in a .S or in .c or in .cpp or .rs or whatever is just an implementation detail
<NicePotato> Do I just plop this into a header and include it in my kernel (plus load it of course)?
<bslsk05> ​wiki.osdev.org: GDT Tutorial - OSDev Wiki
<geist> well, depends on if that does what you want
<geist> does it have the segments you need?
<heat> how about you write your own code?
<NicePotato> I'm not sure what parts I need for my system :P
<NicePotato> what is that emoji eww
<heat> ok you need a null segment, a code segment for ring 0 ("kernel mode"), a data segment for ring 0 ("kernel mode")
<heat> anything else will come later
<geist> okay, so then question: are you writing 32 or 64 bit?
<heat> must be 32 geist, you need a GDT to jump to 64
<NicePotato> 32-bit, but plan on adding 64-bit support in the future
<geist> okay, then yeah you need just 3 entries as heat point out
<geist> there are example ones in the wiki page
<heat> if you understand what traditional segmentation is, consider that in anything remotely relevant you just set up "flat" segments (that span the whole memory)
<geist> under the flat-example thing
<NicePotato> Those are assembly, I'm not sure where I need to put them?
<geist> just somewhere in memory
<geist> can just be part of your kernel, doesn't matter. a global variable
<geist> as long as your kernel is loaded the table is just somewhere in memory by definition
<heat> again, how about you write your own
<heat> you say you understand it, so get to it
<NicePotato> I don't fully understand it
<geist> heat: it's fairly obvious they dont at all, but wasn't going to bust their chops about it
<heat> then get to that first
<NicePotato> I meant I understood what its for
<NicePotato> sorta
<geist> meh, frankly i'm not so sure it's a huge value to grokking it before using it
<NicePotato> maybe I dont
<geist> because it's just legacy nonsense
<geist> you'll eventually want to know it, but frankly it's just a necessary step at this point
<NicePotato> I want to add disk support and keyboard support next. I do need both the GDT and IDT for these, correct?
<heat> I think it's important to free yourself from the shackles of "copying tutorials"
<heat> yes
<geist> agree, but certain things are just boilerplate on x86 frankly
<geist> shit you gotta do to get to the useful stuff
<geist> that you can fill in later
<geist> i dont think this wiki page helps either, because it goes out of its way to try to explain the theory of everythingm and then show these complicated ways to construct it
<geist> vs 'here's a fuckin table for those that just need the same dumb ass table everyone has to build'
<geist> i'd point you at my table but i'm not sure it'll help, since i also do it in a kinda convoluted way that'll just be more confusing
<heat> enjoy
<bslsk05> ​github.com: Onyx/boot.S at master · heatd/Onyx · GitHub
<bslsk05> ​github.com: rust_os/start.asm at master · thepowersgang/rust_os · GitHub
<Mutabah> Darn, got ninja'd
<heat> from 624 to 628
<heat> no, you got AUTOCONF'd
<geist> bingo, thats what you want
<Mutabah> Why two gdt blocks?
SpikeHeron has quit [Quit: WeeChat 3.8]
<bslsk05> ​github.com: lk/gdt.S at master · littlekernel/lk · GitHub
<NicePotato> Wait, I'm confused, how are these loaded into the GDT with those asm functions?
<heat> rtfm
<geist> lgdt
<Mutabah> none of these are functions, they're data
<geist> look at the manual for how lgdt works
SpikeHeron has joined #osdev
wand has quit [Remote host closed the connection]
<heat> Mutabah, second gdt is for 64-bit stuff. I don't have 32-bit kernel segments in the 64-bit gdt
<Mutabah> Ah. You could probably do with comments on the first one that it's 32-bit only?
<Mutabah> Although, you do you
<heat> first is just the bare minimal for bootstrapping myself into a defined protected mode (or into it at all, if coming from SMP wakeup)
<heat> yep, probably
<heat> all that code is slowly getting improved over time
<Mutabah> mine is labelled `smp_init_gdt` - much clearer :)
<heat> yeah
<NicePotato> what is the linear base address?
<geist> hmm, maybe this is easier to grok: https://github.com/travisg/3x86/blob/master/x86/x86.c#L30
<bslsk05> ​github.com: 3x86/x86.c at master · travisg/3x86 · GitHub
<bslsk05> ​github.com: 3x86/x86.c at master · travisg/3x86 · GitHub
<heat> old files such as this one have been through heaven and hell and various amounts of author proficiency
<Mutabah> NicePotato: Linear addresses are after segmentation, but before paging
<geist> that does the GDT stuff in C entirely
<heat> sometimes even frakensteined from various tutorials. which bit me, which is why I'm so anti tutorial
wand has joined #osdev
<NicePotato> geist: so that also has implementation for the IDT?
<geist> yes
<heat> the IDT is not "silly x86"
<geist> note that this code assumes it's already in 32bit protected mode. because otherwise it couldn't be in C already
<heat> it's important to understand that one
<geist> correct. GDT is silly x86. IDT is important
<geist> the main reason you see a lot of GDT stuff in asm is lots of code is setting it up before it gets to protected mode, bvecaus eyou need it to
<geist> or to get to long mode. but if you came out of a bootloader like grub that already started the cpu in protected mode
<geist> then you can use C to switch to 'your' GDT and 'your' IDT
<geist> NicePotato: please do not just copy this code, i'm trying to help you by giving you another eample to look at and try to understand
<geist> since clearly the asm part was tripping you up
<geist> the IDT you'll really need to grok, and you *really* need to grab the intel and/or AMD manual and have it ready
<mrvn> NicePotato: And it's highly recommended that you go to 64bit directly. Forget about 32bit. There is no need to learn all the obsolete bits just to have to relearn gdt/idt/tss again for the long mode bits.
<geist> you cannot rely on the osdev wiki for this stuff
<NicePotato> geist: yes, I won't shamelessly copy and paste, I have looked through almost all of the meaty skeleton and made changes as needeed (like adding cursor support and newline support)
<geist> umm. okay.
<NicePotato> mrvn: I want to support older computers before 64-bit
<mrvn> NicePotato: older like 3 decades old? Anyone that uses one of those should get a new "old" computer, you get them for free basically.
<NicePotato> That's true, I'll look into it
<NicePotato> It took me like 3 hours to figure out the kernel wasn't compiling because of a missing newline character in one of the make.config files (why???)
<geist> you aint seen nothing yet, if you want to build a kernel from scratch
<heat> i remember my couple of days just straight up debugging random heisencrashes under load
<NicePotato> You're correct, i'd count on it. I have been working on my os for over 6 months, and restarted twice
<heat> "geist: that sounds like a tlb issue"
<heat> it was indeed a tlb issue
<heat> :^)
<NicePotato> what's a "heisencrash"
<heat> well it's like a heisenbug but crashes instead
<heat> trademark me, 2023
<NicePotato> well then whats a heisenbug?
<heat> bing it
<NicePotato> I could just look it up
<heat> >what's a bing
<NicePotato> search engine
<geist> heat: oooh yeah? what was it?
<heat> my stack deallocation wasn't doing the TLB shootdown correctly, because the invlpg inline asm had a wrong constraint (so it was shooting down the address of a local variable...), which made subsequent stack allocations on other CPUs pick up the bad TLB entry, which crashed with stupid stacks
<NicePotato> I just realized I have no idea what a struct is in c, no wonder I have no idea what I'm looking at
<geist> heat: ahhh yeah that'll do it. years ago i had a similar TLB issue with reusing virtual slots where a stack was and it is a real heisenbug
<geist> lots of times if i see a odd failire like that around a lot of thread teardown my thought is immediately this problem
<heat> __asm__ __volatile__("invlpg %0"::"m"(addr));
<geist> since kernel stack deletion is one of the few places that usually stress out TLBs in the kernel
<heat> find the bug :/
<heat> yep
<NicePotato> So is a c struct kind of like a lua table?
<heat> erm
<heat> you should figure out your C/C++
<NicePotato> ?
<heat> asking us what a C struct is and trying to compare with lua tables is not the most productive use of everyone's time
<klange> The answer to "is a c struct kind of like a lua table" is "oh boy, you sweet summer child..."
<heat> at that point you're way better off by picking up the k&r book and going through it
<NicePotato> I wasn't asking what a C struct is, I've just never had to use one before
<heat> "I just realized I have no idea what a struct is in c,"
<geist> right, what folks are saying is you really should stop and learn some more basic C stuff before continuing
<NicePotato> It looks quite similarly structured to a lua table
<geist> since you're about to get into the super deep end of needing to know a *lot* about the inner workings of C
<geist> like this is just the tip of the iceberg. you'll need to hav ea good grasp of how C allocates memory, where it puts things, how functions are called, how the stack is used, etc
<geist> static vs global vs local variables, that sort of thing. you really need a solid understanding of that before getting too far
<NicePotato> I have somewhat of a grasp on those
<NicePotato> Not sure how well though
<heat> you just told us you've never used a struct before
<geist> somewhat of a grasp that you dont know how much is correct is insufficient
<NicePotato> yea, I've never had to use a struct
<heat> so chances are your C knowledge is very limited
<NicePotato> probably :/
<geist> anyway, this is fine. you have to start somewhere. just be aware of your limitations and endeavor to correct them first
<geist> go through a full C set of tutorials, books, whatever. make sure you are at least aware of all of the C basic stuff
<geist> thankfully it's a fairly simple language, by modern standards
<heat> k&r boooooooooooooooooooook
<geist> but you'll need to be very proficient with all types of pointers and all of the concepts of memory management behind it
<geist> and we're telling you this so you dont waste a bunch of your time in teh future. trying to dive into osdev without a basic understanding of C is just a waste of your time. it's not productive
<NicePotato> What do you mean by types of pointers?
<geist> exactly.
<NicePotato> like address vs variable pointers?
<heat> if you had a solid grasp of this stuff you'd know what you just said makes no sense
<geist> like, i'm not going to teach you C right now.
<NicePotato> I know, I'm not going to make anyone here teach me C
<geist> 'all types of pointers' is a kinda silly way to say it but wha ti mean is you need to get really proficient on how to use pointers in C. and i mean more fundamenal than 'put a * in front if it' sort of stuff
<geist> and when you learn it you'll realize what i'm talking about. it's more subtle than it looks
<heat> the k&r book was typeset using a whole pipeline of unix commands, including troff
<heat> very retro, I love it
<geist> but this is all good anyway, because learning C is a valuable thing anyway. or at least for the purposes of writing your kernel, assuming you want to do it in C anyway
<heat> I think every C standard pre C23 was also typeset using troff ofc
<NicePotato> Null Pointer, Void Pointer, Wild Pointer, Dangling Pointer is what you mean by types of pointers I'm assuming? https://www.simplilearn.com/tutorials/c-tutorial/pointers-in-c
<bslsk05> ​www.simplilearn.com: Pointers in C: A One-Stop Solution for Using C Pointers
<heat> no
<NicePotato> oh
<NicePotato> whats that k&r book you mentioned?
<heat> bing it
<NicePotato> https://kremlin.cc/k&r.pdf
<geist> what the hell, that tutorial you linked is so stupid
<heat> please don't link to high seas content in the channel
<NicePotato> hmm?
<geist> it's like written by an AI
<geist> with the whole 4 types of pointers nonsense
<NicePotato> ah
<geist> keep in mind that a lot of random crap you find on the net is garbage
<geist> also what heat was saying is dont link to pirated stuff
<NicePotato> I didn't know it was pirated, shoot
<NicePotato> sorry bout thqt
<heat> wdym, the first result is an amazon link
<heat> unless you actually bing'd it and bing is THAT bad
<geist> heat: you should check out that tutorial they linked.it's totally random
<geist> it goes off on some like weird shit about dangling pointers being a 'type' of pointer
<heat> Pointers are like special utilities used by web developers to make it easy to map around in a program code.
<heat> I develop exclusively using C to WASM
<heat> this is somehow worse than geekstogeeks stuff
<NicePotato> I have glanced over the entire of chapter 5 on that k&r book and I still don't understand what you mean by types of pointers. What I thought a pointer was is just a variable with an address in RAM inside.
Vercas has quit [Remote host closed the connection]
<heat> please read the whole book
<NicePotato> oh god
Vercas has joined #osdev
<heat> also"'all types of pointers' is a kinda silly way to say it but wha ti mean is you need to get really proficient on how to use pointers in C. and i mean more fundamenal than 'put a * in front if it' sort of stuff"
<heat> it's just an expression
<heat> it's a small book, 200 pages
<NicePotato> "small"
<heat> yes
<NicePotato> What books do you normally read?!?!
<heat> right now im looking at 2 maths books with >700 pages each, obama's book is also 800 pages long, etc
<heat> the intel cpu manuals are like 4000 all together
<NicePotato> ._.
<geist> NicePotato: yeah again i told you 'all types of pointers' is the wrong phrase
<geist> dont search the exact specific thing. that's entirely the problem. you're tryign to learn the language by just web searching for specific topics
<geist> you really need to sit down and read a book or something that tries to teach you the whole language, topic by topic
<geist> building on previous ones. taht's what books do
<NicePotato> I'm reading through that k&r book
<geist> random tutorials or searching the web for little topics doesn't do taht
<geist> good
<NicePotato> Maybe I'm taking types of pointers too literally?
<geist> you are 100% taking it too literally
<geist> i told you like 3 times to not take it too literally
<geist> it was a typo when i wrote it out
<NicePotato> I'm not sure how not taking it literally would be
<geist> ...
<geist> alright, i'm doing more damage then good here
<geist> i'm going to just shut up
<NicePotato> ok ;-;
<Mutabah> This isn't a C support channel (there's ##c for that :D), but really - pointers are just the address of something, wrapped up in some compiler abstraction
<NicePotato> Yes yes, sorry for going off-topic
outfox has quit [Ping timeout: 248 seconds]
voidah has joined #osdev
gildasio has quit [Ping timeout: 255 seconds]
outfox has joined #osdev
<NicePotato> Now that I understand how structs work, I can understand the GDT stuff better
<heat> read the whole book
<NicePotato> I did
<NicePotato> glanced over everthing
<Mutabah> heat: Re-read their comment :)
<heat> ...
<Mutabah> Oh, wait, are you questioning how quickly they read it... yeah, that was fast for a full read
<NicePotato> I already have a good grasp on some other parts of C, I understood basically all of chapters 1-4
<bslsk05> ​www.youtube.com: The fastest Reader in the world - YouTube
<heat> you sure do mr "i've never used structs"
<voidah> heh ,osdev is not about reading speed, it's about quality writing
<NicePotato> heat: where did you find that video of me
[_] has quit [Remote host closed the connection]
gildasio has joined #osdev
gildasio has quit [Ping timeout: 255 seconds]
gildasio has joined #osdev
_xor has joined #osdev
bradd has joined #osdev
smeso has quit [Quit: smeso]
smeso has joined #osdev
heat has quit [Ping timeout: 256 seconds]
<NicePotato> mats1 excuse me???
<NicePotato> why? why here? why me?
Arthuria has quit [Remote host closed the connection]
mats1 was kicked from #osdev by Mutabah [No.]
mats1 has joined #osdev
<Mutabah> NOBODY click on that link, it's not something you want to see
mats1 was banned on #osdev by Mutabah [mats1!*@*]
mats1 was kicked from #osdev by Mutabah [No autorejoin either]
Terlisimo has quit [Quit: Connection reset by beer]
CryptoDavid has quit [Quit: Connection closed for inactivity]
NicePotato has quit [Quit: Client closed]
Terlisimo has joined #osdev
EineNdamen has joined #osdev
<geist> Mutabah: thanks
<geist> i guess +1 for discord where you can go back and delete a line, huh
<Mutabah> yeah... although that video was uploaded to discord... so they lose a point for making it so easy to host that alongside completely benign content?
mats1 has quit [K-Lined]
EineNdamen has quit [Read error: Connection reset by peer]
EineNdamen has joined #osdev
<geist> -1 discord
EineNdamen has quit [Client Quit]
bgs has joined #osdev
<sham1> Discord is cringe
vdamewood has quit [Quit: Life beckons]
eroux has quit [Ping timeout: 265 seconds]
bliminse has quit [Quit: leaving]
eroux has joined #osdev
GeDaMo has joined #osdev
bliminse has joined #osdev
craigo has joined #osdev
craigo has quit [Read error: Connection reset by peer]
craigo has joined #osdev
danilogondolfo has joined #osdev
alexander has joined #osdev
alexander has quit [Quit: ZNC 1.8.2+deb2+b1 - https://znc.in]
alexander has joined #osdev
gog has joined #osdev
nyah has joined #osdev
<zid> oops, installed my steam client backwards, now I have a bunch of meat everywhere.
slidercrank has joined #osdev
Left_Turn has joined #osdev
Left_Turn has quit [Max SendQ exceeded]
Left_Turn has joined #osdev
Left_Turn has quit [Remote host closed the connection]
Left_Turn has joined #osdev
Vercas3 has joined #osdev
Vercas has quit [Remote host closed the connection]
Vercas3 is now known as Vercas
Turn_Left has joined #osdev
<gog> zid: better than installing it anagramatically and getting teams
Turn_Left has quit [Max SendQ exceeded]
<theWeaver> lmao
<theWeaver> gog is funny :>
Turn_Left has joined #osdev
Turn_Left has quit [Max SendQ exceeded]
Left_Turn has quit [Ping timeout: 248 seconds]
Left_Turn has joined #osdev
Turn_Left has joined #osdev
Left_Turn has quit [Ping timeout: 264 seconds]
bauen1 has joined #osdev
eroux has quit [Ping timeout: 260 seconds]
<kof123> mario kart, castle level: "oh look, free pots, i could use some of those!" <drives into lava> get out of my 8 year old head zid
eroux has joined #osdev
<sham1> Never
Left_Turn has joined #osdev
Left_Turn has quit [Max SendQ exceeded]
Turn_Left has quit [Ping timeout: 248 seconds]
Left_Turn has joined #osdev
Turn_Left has joined #osdev
Left_Turn has quit [Ping timeout: 248 seconds]
bradd has quit [Ping timeout: 248 seconds]
Left_Turn has joined #osdev
Turn_Left has quit [Ping timeout: 248 seconds]
Left_Turn has quit [Client Quit]
SpikeHeron has quit [Quit: WeeChat 3.8]
SpikeHeron has joined #osdev
frkazoid333 has quit [Read error: Connection reset by peer]
gog has quit [Quit: Konversation terminated!]
heat has joined #osdev
<heat> gog
<heat> gog
<heat> gog
<heat> LITERALLY ghosted
<heat> smh my head
<sham1> Let's not get too heated about this
<heat> you're a sham sham
<sham1> Yes
<heat> csgo source 2 coming out omg omg omg
<heat> i'm hyped for anything valve
SpikeHeron has quit [Quit: WeeChat 3.8]
SpikeHeron has joined #osdev
bnchs is now known as stupid_bnchs
eroux has quit [Ping timeout: 246 seconds]
eroux has joined #osdev
zxrom has quit [Quit: Leaving]
mctpyt has joined #osdev
stupid_bnchs is now known as stupid
zxrom has joined #osdev
stupid is now known as stupid_bnchs
tepperson has joined #osdev
stupid_bnchs is now known as bnchs
vismie has joined #osdev
<FireFly> source 2 was released 9 years ago
<heat> that's like yesterday in valve time
<mjg> bur
<heat> omg its richard from richard and mortimer
<mjg> which one is rick
<heat> mortimer obviously
<mjg> iremember trying to watch the first episode some time ago and could not get past first efw minutes
<mjg> "drunk acting" is my pet peeve
<mrvn> What do you mean? *hicks*
<heat> i liked it
* mjg stumbles
<heat> but now it's pretty much dead so
<mjg> heat: genz i take it
<heat> yes it's very gen z
<mjg> is the schtick there that the 2 main characters are in fact the same person?
<heat> no
<mjg> both happy and disappointed
<heat> mjg, why are you not big into non-flamegraph tracing
<mjg> who said i'm not
<heat> i remember I showed you a whole system trace example and you were like "meh"
<mjg> you do realize all *sampling*, while most useful, can only get you so far
<mjg> that was presumably a comment on the thing at hand, not the concept
<heat> ui.perfetto.dev
<mjg> anyhow, tracing or not, one funamdnetal limitation of just collecting some data across the entire run
<mjg> is that you have no idea how it spreads over time
<heat> click "Open Android Example"
<heat> yea
<mjg> does not open
<mjg> lemme try on chrome
<mjg> do you know how big is it?
<heat> yes, it's large
<heat> but loads pretty smoothly here
<heat> so, erm, freebsd moment? nah jk you use linux
<mjg> linux moment
<mjg> oh there you go, got a progress indictaor: 59%
<mjg> so this thing reports stuff over time?
<nikolar> it took a while to load for me kek
<mjg> *that* is not particularly new
<mjg> aight it loadded
<mjg> now lets shit on it
<heat> well the thing you're looking at is a system wide trace of android
<heat> including kernel stuff and other shit higher up the stack
<mjg> good grief the thing i'm compiling psassed -DNDEBUG 4 times
<mjg> so how do i get user/kernel traces from there?\
<heat> there = what?
<bslsk05> ​Netflix/flamescope - FlameScope is a visualization tool for exploring different time ranges as Flame Graphs. (172 forks/2778 stargazers/Apache-2.0)
<mjg> > FlameScope begins by displaying the input data as an interactive subsecond-offset heat map. This shows patterns in the data. You can then select a time range to highlight on different patterns, and a flame graph will be generated just for that time range.
<mjg> 's what i'm talking about mate
<heat> that's cute
<mjg> you want cute here is an idea
<mjg> lock contention flamegraph
<heat> anyway the point is that you can set up the traces how you want them to
<mrvn> mjg: each lock burns hotter the more contention it has?
<heat> the native chrome trace format has support for stack frames attached to random events, etc
<heat> i've been prototyping some tracing stuff locally
<mjg> mrvn: the point is to have "stacktrace" of locks held as you contend on something
<mrvn> so you see where the lock request comes from?
<mjg> here is something which is answerable by "hand", and which could be sorted out by tooling
<mjg> say you took a lock
<mjg> and someone else is waiting for it
<mjg> but you do something expensive
<mjg> there should be an ez way to show what are you "really" waiting for, instead of just saying "the lock mate"
<mjg> in this case, it would show what the owner is doing
<mjg> as a continutation of the flame bar
<mjg> moreover, you could annotate that given thing is done with this or that lock held by having it appear as part of stacktrace
<mrvn> So you want the lock holders stack. That would be a problem for me because the lock holder is always another core and it might change page tables while you try to get the stack trace.
<heat> mjg, anyway I want a nice event timeline graph thing
<mrvn> ==> The current core has to map the stack stack of the other cores process to safely access the stack and that gets expensive.
<mjg> heat: looks like flamescope is the thing mate
<heat> particularly as a way to see how much time I'm spending under preempt-off/irqs-off
<mjg> heat: i have not used it yet though
<heat> no, this is the thing
<mjg> off cpu time profiling does not require specific progression tracing
<mjg> you can go the easy route and get a total count for the workload
<bslsk05> ​docs.google.com: Trace Event Format - Google Docs
<heat> you should take a look at this format
<heat> perfetto can load it and so can chrome://tracing
<mjg> i need the tstacktraces mate, it does not show them
<heat> it can do that, yes
<heat> in fact you can also literally just use this as a thing where you plop your function timings too, ez
<mjg> i'll look into it in the elusive foreeseable future
<mjg> meanwhile i got the slab alloc data from netflix with persmission to share
<heat> noiiiiiice
<heat> do show
<bslsk05> ​dpaste.com <no title>
<mjg> there is a debugging option which multiplies slabs for malloc
<mjg> and they did not turn it off
<heat> ok I can't read this
<mjg> by admission of the guy, he did not get around to it
<mjg> ops lost the header
<mjg> lemme repaste
<bslsk05> ​dpaste.com <no title>
<mjg> the machine has up time of almost 11 days
<heat> i want a nice histogram mr mjg
<heat> ;_;
<mjg> then make one
<heat> seems like malloc-128 is the key slab here?
<mjg> hehe... he
mctpyt has quit [Ping timeout: 260 seconds]
<heat> yeah totally let me install microsoft excel first
<heat> i'll give you all sorts of pie charts after that
<mjg> you may want to awk first
<mjg> splice to one
<heat> i have no idea how to awk
<heat> awk is definitely a unix geezer utility
<mjg> OOH
<heat> it just is
<mjg> did you know dtrace was modelled after awk
<mjg> genz was a mistake
<heat> awk + vim is the one two UNIX HACKER punch
<heat> yes, I know the language is similar to awk
<mjg> you are the boomers of 2000s
freakazoid332 has joined #osdev
<mjg> heat: i expect a power point presentation hre
<heat> mjg, either all tracing buffers are percpu and userspace knows it (and fetches each CPU's buffer) or I'll have a large global buffer and smaller percpu buffers that dump to the global one
<heat> I don't know which one is the best
<mrvn> heat: if you can pin tasks to cores the per core data is much more valuable.
<mjg> userspace should know about per-cpu because per-cpu may be of importance
<mjg> for example i got memset traces for each one
<mjg> which i can replay as they happened on the particular cpu
<mrvn> e.g. pin all proccesses away from a core except the one you want to trace
<heat> but if you tagged every tracing event with the cpu number wouldn't that have the same effect?
<zid> heat are you a full amd fanboi
<zid> can you give me any tips
<mjg> heat: you would *neeD* to do it, which adds to trace size
<mjg> as is you have this bit for fre
<mjg> i'm not saing it is a deal breaker, but i would totes go with full understanding, so to speak
<mjg> make the ulib + kernel compotent a tight dep and hide the split by default for example
<mjg> also note there is a funny problem trying to determine time ordering between events, if you need one
<mjg> perhaps a globally increasing (and serializing :X) id would do it
<heat> zid, i've never had amd hardware in my life, why
<heat> mjg, why is it hard?
<heat> doesn't the monotonic counter work here?
<mjg> it does work but it serializes
<mjg> no problemo on laptop scale
<mjg> oh you mean cycles counter?
<mjg> that still gives some ambiguity so to speak, but that would work for the most part
<heat> why does it serialize?
<heat> rdtscp does serialize the instruction stream but if you don't then I'm fairly sure you can't really have reliable timing there
<heat> like rdtsc alone doesn't work
<mjg> i meant a global counter serializes
<mjg> grabbing rdtscp would not, but you have problem with "ties"
<heat> do I?
<mjg> you can get literally the same value, which one is considered first then
<heat> CPU0 has an event at ts 10, CPU1 has an event at ts 10
<heat> both separate streams of data
<mjg> so if you were to linearize the entire thing, which one happened first
<heat> none
<heat> pick 1 randomly?
<mjg> defo not *randomly*
<heat> but is linearizing the whole thing useful anyway?
<mjg> you want reliable generation from the set
<mjg> sure it is, when you dn't care where someone was running, but what happened across the load
<mjg> for example there is this dtrace script to dump all execs
<heat> this is where I ask you what dtrace does
<heat> like, erm, if the nanosecond timestamp matches up exactly, how is order even relevant there?
<heat> events surely didn't influence each other
<mjg> you take the timestamp when you take the timestamp
<mjg> which is not strictly when the event at hand has happened
<mjg> as in there may be arbitrary delay
<mjg> so i don't know what dtrace is doing there
<mjg> note they had to work with sparc
<mjg> which likely influenced some of the internals
<heat> why is there arbitrary delay?
<heat> doing rdtsc, etc should be done with irqs off
<heat> like: "cli; rdtsc(); write_event(); restore_IF"
<heat> in this case even if you introduce a global counter it won't solve any issue. doing the atomic_fetch_add is still subject to jitter and stuff
<mjg> it is, but you get *some* ordering
<mjg> foo(); bar(); ..... report_that_we_got_here();
<mjg> after bar and before report_that_we_got_here you may get all sorts of shit
<mjg> similarly: report_that_we_got_here(); ... ; foo();
<mjg> there may be eons before you get to foo
<heat> yes
bauen1 has quit [Ping timeout: 268 seconds]
<heat> i understand the problem, but I'm not sure if it matters
<mjg> i'/m saying you want a reliable to way to agree with yourself what the order is
<mjg> even if it happens to be inacurrate in reality
<mjg> so in particular "random in case of a tie" is out
<heat> yes but that atomic_fetch_inc will only be done when writing the event right?
<heat> like, erm, at that point you're just subject to jitter for that right?
<heat> i genuinely can't tell if using a simpler criteria would be better (order by name in lexicographical order)
<mjg> whether you atomic add or cli(); ... or call a webservice, there is said arbitrary delay
<heat> cuz if the exact timestamp matches up, irqs off, and you're racing another CPU for a cache line, sounds like a crap shoot to me
<mjg> dude there may be literally millions of cycles before the event and getting the id/timetsamp/whatever the fuck
<heat> yes, I understand that
<heat> but getting the ID would be part of writing an event here
<mjg> so i'm sayin make sure you got some form of reliable ordering, pretending whatever identifier you got matches relative ordering to others
<mjg> burp
<mjg> look mate, just get something which works
<mjg> i think openbsd has a tracing framework :s
<mjg> they tried to kind-of reimplement dtrace from scratch i think
<heat> oh fuck yeah
<mjg> take a look at this mofer
<heat> my idea is still to add eBPF support and bpftrace my kernel
<mjg> sounds good
<mjg> dtrace -n 'fbt::memset:entry /arg2 > 256/ { @[arg0 & 0xf] = quantize(arg2); }'
<mjg> you get an equivalent operational and we call it even
<heat> but this is good for a generic trace gathering thing where I can also just insert random trace points and not worry with stupidly dynamic ebpf stuff
<heat> naw
<mjg> pft!
<mjg> i'm sticking to sortix then
<heat> you know that requires actual programmability
<mjg> for me that would be the fun part for this problem
<mjg> a dsl which works here
<heat> that would be fun yes, but at that point I would just add eBPF
<mjg> interesting bit: in dtrace they intentionally did not include loops to provably *always* finish probe execution
<heat> and if I add an eBPF interpreter, I'll also write an x86_64 eBPF jitter
<heat> note that I already have the cBPF equivalent but eBPF seems like a good bit of work
<mjg> uh?
<mjg> interestin
<mjg> port dtrace, a weekend project at most!
<heat> no
<mjg> trust me, i used it over the weekend
<mjg> maybe geeksforgeeks have a tracing framwork you can use
<mjg> that's the memmove dudez
xenos1984 has quit [Ping timeout: 248 seconds]
<heat> i'm very well aware of what geeksforgeeks is
<heat> it's like w3schools
<mjg> i mostly was not :X
<mjg> are they related to tutorialspoint?
<mjg> i knew that one
xenos1984 has joined #osdev
<heat> oh yeah thats also a thing
<mjg> back when i used to post on stackoverfow, a victim asked about a problem and ilnked to tutorialspoint
<mjg> i responded saying how crap the site is
<mjg> and giving a better resource
<heat> a victim hahaha
<mjg> SOtwats gave me shit for doing it
<bslsk05> ​github.com: Onyx/spinlock.c at 89e4d0e81e543cbd6c9a3978f546feddf8840e7f · heatd/Onyx · GitHub
<mjg> apparntely you either answer the question or stfu, opinions like the above are offtopic
<mjg> fucking idiots
<heat> vintage Onyx spinlocks
<heat> spinlock_t * lock omg omg the *
<mjg> wait that does not even work?
<heat> it's broken yes
<mjg> you can submit to tutorialspoint
<mjg> the bug aside, if (lock->lock == 1) -> while (lock->lock == 1) is primo stuff
<heat> if (lock == 1) { spin_and_yield(); } test_and_set();
<mrvn> mjg: opinions are expressed by down votes
<sham1> 👎
<sham1> Now that's a down vote
<mjg> SO, openbsd, netbsd and solaris are part of the stuff i no longer look at
<mjg> life's better
dude12312414 has joined #osdev
<heat> https://github.com/heatd/Onyx/blob/4bcc27fe7eae67fc473bf7a2ab94a749d962c60c/kernel/arch/x86_64/spinlock.S#L3 2017 is where I really started to get the hang of this osdev thing
<bslsk05> ​github.com: Onyx/spinlock.S at 4bcc27fe7eae67fc473bf7a2ab94a749d962c60c · heatd/Onyx · GitHub
<heat> 2015 and 2016 were just a crapshoot
<mjg> you *start* with pause?
<mjg> ffs man
<mjg> i think i had seen this anti-pattern before
<mjg> have you copied this from somewhere?
<heat> i don't think so
dude12312414 has quit [Client Quit]
<mjg> xchg to unlock is also great
<mjg> ye ok mjg@
wand has quit [Ping timeout: 255 seconds]
<heat> it's not good but it does the job innit
<mjg> the old unix maxim, innit
<mjg> "ultimately gives you the correct result so who gives af"
<mrvn> heat: do you really need the frame pointer?
<kof123> "intentionally did not include loops to provably *always* finish probe" <dispatches from another channel> wasm apparently says tail call stuff must use some limited resource that eventually exhausts, else people would depend on a platform that did not do this (at least how i read it)...
<mrvn> kof123: what platform uses up resources on a tail call? That's just a goto
<kof123> i dunno, long ago i remember a teacher talking about it, some compilers can do it, some ancient ones couldn't
<heat> bpf has historically been unable to jump backwards
<mrvn> kof123: sure. if your compiler does not do tail calls then you don't get tail calls.
<mrvn> kof123: That's a compiler feature, not a language feature.
<heat> i think current eBPF can, given it provably returns. not 100% sure though
<kof123> dont disagree, just the comment made me think of that
<mrvn> wasn't eBPF using llvm?
<kof123> for me you'd probe at run-time, or a constant, or whatever. "duck"-ish
<kof123> i mean, there is a point where i say "assume the developer has a clue"
<kof123> rather than try to dictate every little thing
<mjg> for some like kernel-side tracing with user-supplied prog it is a reliability issue
<mjg> here user == root
bnchs has quit [Read error: Connection reset by peer]
<mjg> invalid memory accesses are already caught
<mjg> instead of assuming the user clearly would not make a mistake
<kof123> certainly, i see the use of such things depending on the situation
<heat> mjg, security issue too
<heat> you can submit BPF programs as !root, you can also submit BPF programs as root on secure boot/locked down kernels
* kof123 .oO( unstable radioactive duck glowing different colors, there's surely a comic there )
xenos1984 has quit [Ping timeout: 256 seconds]
<mrvn> Supermans Rubber-Ducky better not be made with Kryptonite.
gog has joined #osdev
wand has joined #osdev
<heat> gog
<sham1> How can it Kryptonite if it is rubber?
<mrvn> sham1: you think rubber-duckies are 100% rubber?
<sham1> Well the non-air portions, I'd hope to be rubber. Otherwise it's false advertising
<gog> heat
<mrvn> sham1: and what color does rubber have?
<heat> hi
<sham1> Oh so you'd put Kryptonite in the colours? Well that's douchy
<mrvn> how else do you get green, red and black?
<sham1> Probably by using something that isn't Kryptonite
<bslsk05> ​illgetdrivethru.com: What are the different types of kryptonite? | I'll Get Drive-Thru
<gog> hi
xenos1984 has joined #osdev
<Ermine> hi gog
<gog> hi
<Ermine> may I pet you
<gog> yes
* Ermine pets gog
* gog pr
<heat> gog pull request
<gog> decline
<heat> heat has requested changes
<gog> decline
<Ermine> alias gog=git
<heat> gog push
<sham1> Oh my
eddof13 has joined #osdev
bnchs has joined #osdev
bnchs has quit [Remote host closed the connection]
<heat> mjg, what stops the CPU from detecting a e.g 4 64-bit load+store and doing it as a 32-byte one?
<mrvn> a 4 64-bit load+store?
bauen1 has joined #osdev
<heat> yes, 4 64-bit load+stores
<heat> so 32 bytes in 8 byte chunk
<heat> chunks
<sortie> <mjg> i'm sticking to sortix then
<sortie> The safe choice, I applaud
<sortie> heat, you self-hosting yet?
<heat> no
<sortie> Such a shame
<heat> i'm busy bikeshedding sortie, leave me alone!
<sortie> What color u want teh man pagez
<mrvn> heat: and you think the CPU will do 8 32-bit load+stores?
<heat> yellow and pink
<heat> if it looks good, please do tell, so we can change that
<mrvn> oh wait, never mind. I read 32-byte as 32-bit.
<mrvn> I'm not sure what prevents that on x86/x86_64 but on ARM you specify that in the cache attributes. Write back, no write combine, etc.
<mrvn> I got my power adaptor and SSD today and now wanted to start playing with my Rock 5B. Now I notice that 1) I don't have an USB-C / USB-C cable and 2) neither the Rock 5B nor SSD came with the screw you need to hold it in the M2.key slot.
<mrvn> *schnief*
robem has joined #osdev
wootehfoot has joined #osdev
<mjg> heat: instruction contract, if you will
<mjg> heat: aligned 8 byte stores are guaranteed atomic
<mjg> heat: same goes for loads
sinvet__ has quit [Read error: Connection reset by peer]
nortti has joined #osdev
<mrvn> mjg: combining them to 32-byte would still be atomic
<mjg> wait i misread, wtf is that question
<mjg> 32 *BYTE*?
<heat> yes
<heat> like the CPU would notice a 32-byte copy as a bunch of 8-byte ones and combine it all into a 32-byte op
<mrvn> Not sure how you get 32 byte. I assume that's some SSE256 operation.
<mjg> he mans the 4 * movsq loop i presume
<heat> yep
<mjg> for a case like that, i don't think anything prevents it
<mjg> but i don't know if any cpus do it like that either
<mrvn> Does it even matter? It will be combvined into a cacheline.
<heat> it would be a cheap win I think
<heat> AVX-but-without-AVX-and-rep-movsb
<mjg> i know enough about cpus to know that there are no clear wins mante
<mjg> when it comes to wtf is going on in the internals
<mjg> in fact i know enough to know i don't understand almost any of it
<heat> i wonder if there are limitations in the insn pipeline such that this is not practical
<mjg> a big wig from intel once said that cpu internally executes shit in an inherently random manner
<mjg> but people expect reliable results from it
<mjg> on that note you would think 'rep mov' et al would be a clear cut win
<heat> yeah
<mjg> you tell the cpu what you want to do, it can optimize however it is best
<heat> apparently not
<mjg> right
<heat> cpus are weird man
<nortti> isn't it optimized pretty well (some startup overhead, but quick copying speeds) on more recent core i?
<mjg> nortti: somewhat, but no
* heat holds mjg back
* CompanionCube waves at nortti
<mjg> tl;dr handrolling ops are not going anywhere anytime soon
<mjg> fsrm or not
<heat> moderately-recentish rep movsb is fast-ish at higher speeds but it still has weird limitations, like on my kabylake where it seems to use SSE2 stores internally
<heat> and is slow as molasses for small sizes
<heat> i think FSRM reduced the turtle-speed sizes to < 64 but I don't have hw to test that on
<mjg> here is homework for you heat: check pre-erms routines in various kernels
<heat> (FSRM being an even more recent optimization)
<mjg> it is all rep {m,s}tosq -> rep {m,s}tosb
<mjg> the back to back rep is so atrocious it is not even funny
<heat> my favourite instruction is rep mtosq
<heat> i mean, linux seems/'d to have a good handrolled manual loop originally
<mrvn> heat: maybe some cpu will see the 4 movq and fuse them into a single uops.
<mjg> no opinion
<mjg> they did move away from it
<mjg> so
<heat> yes because haha erms good
<heat> kneejerk reactions to intel claims
<mjg> er... molasses speed
<mjg> fun fuct is that intel claims rep stos + 4096 buf size + page-alignment rolls with some internal speed up
<mjg> i failed to measure any though
<heat> should do a tlb shootdown :v
<mrvn> mjg: any slowdown if you rep stos 4095 bytes?
<mjg> no idea, never tried
<mrvn> then what did you compare it to?
<mjg> rep stosq
<mjg> that above was b
<mrvn> you sure they ment stosb and not stos* in general?
<mjg> yes, it was one of the benfits of erms
<mjg> and intel even migrated linux clear_page to use it
<mjg> now you could assume stosq "translates" to erms internally if cpu supports the bit, but then see above
zxrom has quit [Ping timeout: 248 seconds]
<mjg> it's all weird shit where basic ideas how things work fail
valshaped3 has joined #osdev
valshaped has quit [Ping timeout: 255 seconds]
valshaped3 is now known as valshaped
<moon-child> I wonder if it would be reasonable to have the dram hardware support page zeroing directly
<mrvn> moon-child: and if the ram is interleaved and you only want to zero half a page?
dude12312414 has joined #osdev
<mrvn> FYI: the hardware that does this kind of thing is called DMA engine.
dude12312414 has quit [Remote host closed the connection]
zxrom has joined #osdev
theboringkid has joined #osdev
paulbarker has quit []
tepperson has quit [Quit: Client closed]
<mjg> moon-child: except most of the time you ant the zeroed page to stay in cache, if already there
<mjg> so that would be an addition
wootehfoot has quit [Read error: Connection reset by peer]
<mrvn> mjg: the cache could have a "zero page" operation too
<moon-child> ideally you would just rename the cache lines
<mjg> so as i said ram thing would be an addition
<moon-child> azul cpus did this per allocation. But whole page at a time is more iffy
<mjg> to the core operation
<mjg> preferably the semantics would be: tell ram to zero, whatever lines from that page are in cache, zero them too. fuck the rest.
<mjg> as don't evict any of these lines, but don't bring in new lines either
<moon-child> if everything (or almost everything) is in cache, then you wouldn't want to bother telling the memory anything
<mrvn> I was about to suggest it bring in new lines if it doesn't have to evict anything. But when would that ever happen?
<mjg> sure
<mrvn> moon-child: telling the ram to zero a page in a single bus transaction would be better than having the cache write back 4k zeroes
<moon-child> that's why I said 'IF everything (or almost everything) is in cache'
<mrvn> You tell the ram to zero the page and the cache to zero out any cache entries present and mark them as not-dirty.
<mrvn> The alternative would be to evict any cache entries and later read back zeroes. That would be wasteful.
<moon-child> if there's only one cache line not in cache, you're better off just telling the ram to zero that one cache line, since otherwise it would waste a bunch of work zeroing the rest of the page
<mrvn> moon-child: haeh? You still have to zero the ram anyway
<moon-child> yes but, like mjg said, the thing that are already in cache you want to keep there
<moon-child> since you're goign to use them
<mrvn> but you don't want to have to write them back to ram
<moon-child> and most likely write something other than zeroes
<moon-child> before you flush them again
<moon-child> the contents of the cache are different from the contents of memory. when something is in cache, the version of it that is in cache is the canonical version, and the version in main memory may be stale. That's how memory works
<mrvn> and my way would have both cache and ram in sync.
<heat> wasn't there a cache line zeroing insn on arm64?
<moon-child> mrvn: yes and there's literally no reason to do that
<moon-child> it's wasted work
<mjg> heat: amd has one (clzero) but it is using nt stores
<mrvn> moon-child: there is every reason for it. You can not leave the old contents in the cache. So the choice is to evict the cache line or zero it. And if you don't tell the ram about it then the cache has to mark the cache line as dirty and eventually writes it back. Then you have gained nothing over just writing zeroes to the page.
<mjg> so it's liek, cool story bro
<moon-child> mrvn: yes you did
<moon-child> you zero the page
<moon-child> then you hand the page to the user
<moon-child> the user writes some stuff which is not zero to the page
<moon-child> all this without touching main memory, ever
<moon-child> then finally it gets evicted, writing out to memory the user's new data
<moon-child> not zeroes
GeDaMo has quit [Quit: That's it, you people have stood in my way long enough! I'm going to clown college!]
<mrvn> moon-child: maybe he does, maybe he doesn't. That's later.
<moon-child> but you bet that he does, because that's faster in practice than betting he doesn't
<mrvn> moon-child: and you lost nothing by having the ram cleared the page and the cache entries marked clean.
<moon-child> yes you did
<moon-child> because the ram has to do extra work
<mrvn> yes, draw the zero-page line high on the page for one cycle. Big deal.
<mrvn> That's as much work as writing out one cache line.
<gog> hi
<heat> mjg, if amd got something completely new in ryzen it's probably because there's an arm64 equivalent
<bslsk05> ​developer.arm.com: Documentation – Arm Developer
<heat> there it is
<mrvn> moon-child: One use case for this would be mmap, paging in filesystem data. So a security conciencus kernel would zero the page before telling the FS to read data into it so no information leaks. Keeping the zeroes only in cache would mean having to write them all out.
<mrvn> Actually my dream operation would be: zero page in ram, evict cache lines and shootdown the TLB (across all cores).
<mrvn> unmap(page) in hardware
rorx has quit [Ping timeout: 255 seconds]
<moon-child> nah I want lazy shootdowns
<mrvn> what makes it lazy?
<moon-child> for standard posix munmap, you have to actually munmap right away, since acceses that happen after have to fault
<moon-child> but for most applications you don't actually need this
<moon-child> so better is if you can let each core release the page when it feels like it
<mrvn> does it? isn't access after munmap simply UB?
<heat> i've thought about that for a good bit
<heat> doesn't work, huge security hole
<moon-child> if you're lucky, memory pressure is low, and all the cores release the page before you need it again
<moon-child> otherwise, you do the shootdown later when you need to reallocate the physical page
<heat> it would *in theory* work in a system with root and nothing else
<heat> oh, yeah, that's not my idea
<moon-child> heat: how so? I mean, I guess it makes some categories of use after free ever so slightly worse, but doesn't seem like a huge deal?--oh, ok :)
<heat> my idea would be to do lazy TLB shootdown such that munmap doesn't TLB shootdown but rather mark it as "MAY USE, BUT SHOOTDOWN FIRST"
<mrvn> moon-child: I could live with munmap needing an msync() before SEGV is guaranteed.
eddof13 has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<heat> and then when you first get the chance to accidentally shoot it down (in a cr3 switch, etc), you ""shoot"" all those down in a batch
<moon-child> mrvn: well the other thing is
<heat> but locally only.
<moon-child> this would integrate naturally with a mechanism like io_uring
<moon-child> which is already async
<heat> so in theory there would be no need for a shootdown ever, but this doesn't work since even if you restrict it to root, it's a security hole in systems where root isn't all-powerful (Windows, anything with secure boot/lockdown)
<moon-child> ideally, if buffer size tuning is right, a program can go its whole life w/o waiting for mmap or munmap
<mrvn> moon-child: you could also reuse pages for the same process without unmapping.
<moon-child> yes! Was just about to mention that
<moon-child> can also skip zeroing, though you would need a new mmap flag to ask for it
<heat> there's a flag to skip zeroing in mmap
<heat> linux one at least
<mjg> heat: btw ppc also has a 'zero a ton of mem' instruction
<heat> it will just tell you to fuck off on systems without the CONFIG_BLAHBLAH_INSECURE or whatever they called it
<heat> mjg, ppc best arch
<mjg> itanium does not? HEHE
<moon-child> heat: hmmm looks a bit different though?
<mjg> skip zeroing in mmap?
<mjg> ll
<moon-child> manpage says 'because of the security implications', so presumably it doesn't do it only if it's your memory
<mjg> not only that's not secure, but software actively rleies on it being 0
<heat> MAP_UNINITIALIZED "Don't clear anonymous pages."
<mrvn> moon-child: I see in the manpage "The munmap() system call deletes the mappings for the specified address range, and causes further references to addresses within the range to generate invalid memory references." but where in POSIX does it say that access to unmapped memory generates an invalid memory reference at all?
<mjg> has to be some pretty serious corner case
<mjg> curious how that came to e
<mjg> then again, linux claims to run on systems without mmu
<heat> embedded
<mjg> that much i figured, but what exactluy
<heat> moon-child, what is "your memory"?
<moon-child> most of the time it's not actually useful to rely on its being 0. Usually you malloc something and then initialise it
<moon-child> heat: memory that your process recently unmapped
<mjg> jemalloc depends on it
<heat> well, yes, linux does not have that
<mjg> and glibc uses it optimize calloc
<heat> if you did add it, you already have a flag
<moon-child> mjg: sure, but proposal is if application asked to skip init, and was designed to work with that instead, you might go faster
<mrvn> malloc should use MAP_UNINITIALIZED while calloc should not (when allocating large blocks where you mmap directly)
<heat> and tbf adding this seems like a locking nightmare to me
rorx has joined #osdev
<heat> hmm, maybe not so much
<heat> i guess it's a similar-ish operation to OOM-kill
eddof13 has joined #osdev
<mjg> moon-child: not saying no
<mjg> maybe geeksforgeeks has a good page zeroing scheme
<mjg> for example it could start with memcmp over the entier thing, maybe it is already zero?
<mjg> for (i = 0; i < PAGE_SIZE; i++) if (page[i] != 0 /* fix that byte! */ page[i] = 0; }
<mjg> avoids dirtying lines which were already zero!
<mjg> maybe i should submit over there
<kof123> im pretty sure they would int i = 0; that's insecure to use something already declared
<kof123> encapsulation, ppl
<kof123> i? surely index or Index, it'll never pass the strict quality control standards
SpikeHeron has quit [Quit: WeeChat 3.8]
eddof13 has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
SpikeHeron has joined #osdev
eddof13 has joined #osdev
<kof123> *inDex if you want to climb the ladder quickly
Lumia has joined #osdev
_xor has quit [Ping timeout: 255 seconds]
_xor has joined #osdev
meisaka has quit [Ping timeout: 268 seconds]
<heat> INT mIndex = 0; mIndex < gPageSize; mIndex += 1
<heat> mjg, that's SMP-optimized memset
<heat> lgtm
<mjg> jokes aside
<mjg> it would make sense to add a debug option to pagezero
<mjg> to check if perhaps the page is already zeroed
eddof13 has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<mjg> there may be a lol caller which for example gets a page guaranteed zeroed
<mjg> and then zeroes it again on its own
ThinkT510 has quit [Quit: WeeChat 3.8]
<heat> mjg, how good is dflybsd's smp performance?
<mjg> no
eddof13 has joined #osdev
<heat> thank you
<heat> on a rando hn post: "DragonFlyBSD is slowly but steadily becoming more attractive as a Linux/FreeBSD alternative. 4.6 has good support for various newer Intel GPUs, hammer is stable and the SMP and Networking performance is stellar."
<mjg> to dfly credit, *some* stuff scales ok
<mjg> but most of the claim to fame is a 2012 benchmark
<heat> this was in 2016
<mjg> when people claim it scales, they reference the 2012 bench
<mjg> what dfly is doing better than freebsd is pmap
<mjg> and nullfs-equivalent
<mjg> past that i think it is all worse, markedly so at times
ThinkT510 has joined #osdev
<mjg> the shit i mostly worked on -- path lookup -- is way better on free
<mjg> and as noted some other times it even used to beat linux, until a recent change
<heat> why is there no openbsd in that comparison dear freebsd developer matthew garrett
<heat> are you hiding something
<mjg> i think i tried
<mjg> it was hitting the wall pretty hard
<mjg> their syscall handling used to have a scalability problem
<mjg> they had a global call counter
<mjg> :p
<mjg> to give you a glimpse
<heat> when you do you write your own OS for a performance showdown with me
<mjg> beat dfly and i'll think about it
<mjg> let's pick a bench
<heat> i think I can beat netbsd on most metrics except usefulness
<mjg> L]
<mjg> do you have any means of getting a machine to boot onyx on more than your laptop
theboringkid has quit [Quit: Bye]
theboringkid has joined #osdev
<heat> probably boots on cloud but I really haven't tried
<heat> and should depend on the cloud provider
<mjg> you could use tmpfs to avoid any virtio problems
<heat> I can totally do GCP rn BUT I have no disk yet (no virtio scsi support)
<heat> I know amazon has weird amazon-specific devices I don't have docs to except the typical linux driver
<mrvn> heat: I have the fastest booting OS. The functionality is somewhat limited to saying "Hello World" but who cares? :)
theboringkid has quit [Client Quit]
theboringkid has joined #osdev
theboringkid has quit [Client Quit]
theboringkid has joined #osdev
theboringkid has quit [Client Quit]
theboringkid has joined #osdev
theboringkid has quit [Client Quit]
theboringkid has joined #osdev
<mjg> here is a very serious comment
<mjg> today i would not recommend anyone uses freebsd without a good reason
<mjg> not that there are many takers
<mjg> on that note, dragonfly is in ever worse position
<heat> have you told whoever's paying your bills that
<mjg> what if they have a good reason
<mjg> to use it :X
<heat> sounds like "we can't bother to upstream changes and/or do GPLv2 evasion"
<mjg> no
<mjg> majority is upstramed
<heat> actually modules can be proprietary so I ran out of good reasons
<mjg> and in fact any new work goes is encouraged to go usptream first
<mjg> erm
<mjg> s/goes//
<mjg> hell, the main product now tracks the dev branch
<mjg> so ye, plenty of reason
<mjg> but the point was, dawg, when people make comments about non-linux systems, they are bullshitting most likely
<mjg> put it in the same bucket as people romanticizing templeos
<heat> I romanticize FreeBSD and NetBSD as well
<mjg> at best, in certain corner cases, you are not actively behind
<heat> they are venerable, undermanned systems
<heat> no comment about the other one
<mjg> well i literally just now wrote a patch for freebsd vm
<mjg> and fuck me, how retarded that code is is beyond me
<heat> and truth be told my experience with FreeBSD has been positive
<mjg> here is a statement for you
<heat> except the fucking sysctl code, my man that is horrible
<mjg> anytime you meet someone who is hardcore on $system-of-choice
<mjg> they will have a story how $popular-alternative did now work for shit
<mjg> they switeched to $system-of-choice and it's been a smooth ride ever since
eddof13 has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<mjg> both $system-of-choice *and* $popular-alternative could have been any of: free/net/openbsd, random linux distros
<heat> i'm hardcore on windows
<mjg> this is for < 2010
<mjg> i literally got people telling me a linxu router could not stay up
<mjg> they put openbsd in there
<mjg> et voila
<heat> have you ever struggled to install a controller's driver for 2 hours? yes? you're using linux
<mjg> worked like a charm
bgs has quit [Remote host closed the connection]
<heat> boot up windoze, connect the controller, open your game, have fun
theboringkid has quit [Ping timeout: 248 seconds]
<mjg> interestingly these stories are probably mostly true, but they ignore the later realities
<mjg> that the thing which probably worked great initially eventually also started crashing/whatevr
<heat> how old are you
<heat> you're either 79 or 30
* moon-child uses freebsd desktop
<mjg> 37
<mjg> moon-child: lol @ you mate
<moon-child> :<
<mjg> tbh i probably would if i did not need wifi
<mjg> and suspend
<mjg> :)
<moon-child> largely just works; some annoyances, but linux also pulled some fuckery
<mjg> right
<kof123> i used netbsd on a laptop for a long time because it was the only of the 3 i hadnt used before
<kof123> im not saying there was anything behind that decision
<mjg> moon-child: perhaps you would be interested in fixing automemcpy in llvm
<moon-child> desktop not laptop. I think my old laptop ran freebsd and wifi at least was ok, idk though. Current laptop is macbook
<kof123> open had pf, but that got ported
<mjg> moon-child: and benching some code
<mjg> moon-child: to finally get a sensible simd memcpy for libc?
<heat> but eh freebsd path walking is fuckin mint lads
<mjg> kof123: that's liek 2005 news? :)
<mjg> kof123: what year was it
<kof123> :D
<mjg> heat: which one dwag. the *locked* code is pretty... not pretty
<moon-child> mjg: hmm maybe
<moon-child> ifunc somewhat sexier
<moon-child> can avx512 then
<mjg> who says no ifunc
<kof123> im just saying some of us are just .......when choosing the lesser evil, choose the one i havent tried before
<heat> freebsd has ifunc right?
<mjg> it does
<nortti> < mjg> tbh i probably would if i did not need wifi < mjg> and suspend ← my main computer around 2015 was an eeepc 701 running freebsd. recently booted it up again and realized I had no cpufreq on there lol
<heat> well there you have it
<moon-child> oh, it does? I thought it didn't
<mjg> moon-child: toes ifunc, but what to put in the ifunc
<mjg> nortti: solid
<nortti> (though unsure if that was just me not having enabled something I needed to enable)
<moon-child> oh GNU_IFUNC hits in rtld
<mjg> ifunc is actively used in rtld
<mjg> it's almost all there mate
<mjg> you find a good memcpy to use, i'll fix the missing bits
<mjg> the thing to do is to make sure pre-ifunc-resoultion code does not hang
<mjg> add rtld_memcpy_early et al to use instead
<mjg> which would use a "plain" variant
* moon-child nods
<mjg> really, the real work here is to get a defensible memcpy
<heat> freebsd ending up using llvm-libc's stringops would be the most lol moment ever
<heat> the UNIX oldies would have a stroke @ seeing C++ in libc
<mjg> why
<mjg> fair
<heat> it's also a complete bullocks roullette
<heat> will it generate good code? who knows?
<heat> if it doesn't, just patch the compiler like normal people do
<moon-child> isn't freebsd still using c89 mainly?
<mjg> heat: i totes don't want that state
<moon-child> can't decl where you want
<mjg> what i would want is to experiment, find the right routine
<mjg> and then ship the asm as found there
<mjg> moon-child: c99-ish in kernel, userspace is a hugely mixed bag
theboringkid has joined #osdev
<mjg> and to be clear, while technically memcpys at generated in automemcpy may happen to contain the best overall variant
<mjg> they *lack* important cases
<moon-child> oh btw, the avx512 thing is kinda rubbish
<mjg> it probably is
<mjg> i think they focused on sizes < 128
<mjg> anything past that seems yolo'ed
theboringkid has quit [Client Quit]
<mjg> they have a jump table(!) for 1-3 which i'm confident is pessimal
<moon-child> if you do a full-width 512-bit op, it takes a few 100k cycles to warm up the 512-bit register file, during which time everything is emulated with 128-bit ops. And then on older hardware it clocks down
<moon-child> you can still get a speedup from avx512 with 128-bit ops
<moon-child> don't want to use the full-width ops for strings functions because it slows everybody down if the userspace app isn't otherwise using them
<mjg> i would say just avx2
<heat> "during which time everything is emulated with 128-bit ops. " LOL
<moon-child> but what would be nice is if you could use the 512-bit ops and tell it to emulate with 128-bit ops but _not_ spin up the 512-bit thing if it isn't already
<mjg> really, for a full setup, i woudl expect a win over sse2 first and foremost
<moon-child> as you still save on dispatch that way
<moon-child> mjg: avx2 same thing
<mjg> for the sizes < 256 or so
<mjg> really? that's super shit
<mjg> curious what's glibc is doing in that ase
<heat> avx2 same thing?
<moon-child> I think avx2 doesn't clock down
<moon-child> but it does take a while to spin up the register file
<heat> mjg, what case?
<mjg> look mate, if you get automemcpy operational, get some real memcpys to test
<moon-child> I think glibc just yolos it with wide vectors
<mjg> i can plug them int oa real workload and we will see
<mjg> but first i need candiates which at least look good in a bench
<mjg> so really, are you up to it or not
<sortie> $ curl -s --head https://sortix.org/ | grep -Ei 'server:'
<sortie> server: nginx/1.23.3 (Sortix)
<bslsk05> ​sortix.org: The Sortix Operating System
<sortie> ^^ I migrated the website to running natively on nginx :)
<moon-child> meh probably not
<moon-child> tbh
<mjg> moon-child: :p fucker
<moon-child> I'm spreading myself rather thin lately
<mjg> same problem mate
<moon-child> :)
<mjg> that's my page zeor!
<nortti> heat: huh I thought it did 64-bit if stuff was aligned
<heat> oh wow
<heat> what is this
<moon-child> 64-bit it doesn't even matter if you're aligned or not
<heat> it's doing very funny type punning
<mrvn> The Rock 5B is a bigLITTLE cpu (RK3588 - 4x Cortex-A76 + 4x Corctex-A55). Is it normal they have the exact same features and bogomips? https://paste.debian.net/1273307/
<bslsk05> ​paste.debian.net: debian Pastezone
<sortie> heat: BYTES GO BRRR
<mjg> heat: URL
<bslsk05> ​gitlab.com: libc/string/memcpy.c · master · sortix / Sortix · GitLab
<sortie> My memcpy does try to do nice aligned copies :)
<heat> sortie, there's a may_alias attribute that does skip that
<sortie> What do you mean? may_alias?
xenos1984 has quit [Read error: Connection reset by peer]
<moon-child> 'Abort on overlapping memcpy' nice
<mjg> sortie: you can name your main routine "memcpy_slow"
<heat> yes, may_alias tells the compiler that you're type punning with that
<klange> sortie: SMP|aarch64|some-other-thing-I-have-because-my-priorities-are-different when?
<moon-child> sortie: aligned vs unaligned doesn't really matter for 8 bytes thingies
<moon-child> you're better off skipping the dispatch
<heat> mjg, reminder that you're not bantering me this time around
<heat> sortie, but that's UB
<heat> erm, moon-child*
<mjg> real q: can you write real-world libc without UB?
<moon-child> heat: it's also ub if they are aligned
<heat> atm I don't think so
<moon-child> just more ub if misaligned
<heat> moon-child, not with may_alias
<mrvn> What was the command under linux to output the device tree?
<mjg> lspci -v?
<mjg> or so
brunothedev has joined #osdev
<mrvn> mjg: not pci, the device tree: proc/device-tree/ in usable.
<mjg> oh, sorry
<heat> sortie, actually I really think you're hitting UB in memcpy
<sortie> heat, mjg, moon-child: Yeah my memcpy is just meant to be well defined and not unreasonably slow. I have put in almost no effort in optimizing it.
<heat> your union type punning isn't really working here
<brunothedev> hey guys, for a generic gpu driver, what should i do? I don't want to implement a text-only mode driver, and i want a generic driver that works on every i686/x64 machine, should i implement a VGA driver or there is alternative?
<sortie> Yeah I think I later learned unions aren't actually defined like that (fuck you C)
<klange> There is no such thing as a generic GPU driver, and you will not write a GPU driver.
<heat> sortie, as it stands you're just doing type punning on the pointer and not the contents
<klange> This is a pedantic point, but a GPU is a very specific kind of device and should not be confused with a display controller.
<nortti> brunothedev: if you want modesetting, you're going to need to have card-specific drivers. depending on how you're booting your bootloader (or UEFI) may be able to arrange a framebuffer for you though
* sortie . o O (I can use memcpy to do the type prunin--- wait)
<sortie> klange: 1.2 :)
<klange> sortie: when's 1.1?
<heat> sortie, if you slap __attribute__((may_alias)) on your pointers it should be well defined
<sortie> klange: I'm finishing up 1.1, like, for real, it's happening, legit, tell your friends
<brunothedev> oh okay, just something that i can draw pixels at the screen to write something like the linux framebuffer console, any names, wiki pages, docs etc
<heat> but ofc a GNU C thing yadda yadda
<sortie> heat, oh neat, file an issue? Or send a MR?
<klange> sortie: i hear sortie's finishing up sortix 1.1!
<heat> i'll file an issue
<klange> sorry you may be my only friend
<sortie> klange: HE WHAT
<klange> here at least
<sortie> heat, sick burn
<sortie> klange is my friend too
<sortie> klange, unclear about the exact 1.1 timeline but maybe hoping for something late spring or early summer at this rate
<sortie> klange, I literally have just merged my network stack and ssh. Officially. Crazy, I know. Took 7 years.
<mjg> do you have smp yet?
<sortie> mjg, just spawn two Sortix VMs and load balance :D
<mjg> is that cloud multiprocessing?
<sortie> It sure is
danilogondolfo has quit [Remote host closed the connection]
<klange> ah but i have 24 cores, so I'll need to run 24 VMs
<sortie> klange, oh right, I do also have that GUI prototype of mine that I gotta make a little more functional, that also count as as a some-other-thing-I-have-because-my-priorities-are-different
<sortie> Probably will include that in 1.1
<sortie> Other Sortix news: The official Sortix nightly releases will also be built on Sortix every night as of tonight.
<mjg> o/
<sortie> That's possible now that I merged my ssh port
<mjg> how many oses go to fully slef-hosting stage?
<mjg> you joined the rank of templeos mate
<brunothedev> sortie: lol is it unix-like?
<klange> probably quite a few more that don't use gcc or llvm
<sortie> Fun fact operating systems like GNU/Hurd, Minix, Haiku, etc. don't self-host their websites last I checked
<mjg> fun fact: openbsd did not at some point either
<mjg> :)
<sortie> brunothedev, yeah, per sortix.org (running on a Sortix server) “Sortix is a small self-hosting operating-system aiming to be a clean and modern POSIX implementation.“
<sortie> OpenBSD is pretty good at self-hosting their infra these days tho
bradd has joined #osdev
<brunothedev> sortie: i wanna to achieve a similar result too
<mjg> sortie: no idea, i would expect that
<mjg> the problem was way back in the day
<heat> sortie, it's GNU <space> Hurd
<klange> I updated my historic screenshot album with some new stuff. https://gist.github.com/klange/f427a551af5f2f8b3c9ef80687883fcf
<bslsk05> ​gist.github.com: 12 Years of ToaruOS · GitHub
<heat> it's a GNU project mr sortie
<heat> also, can it be considered an operating system?
<klange> GNU/HURD is the OS, GNU HURD is just a kernel ;)
<brunothedev> hurd is just the kernel
Lumia has quit [Ping timeout: 248 seconds]
<heat> no, GNU is the OS, GNU hurd is the """kernel"""
<nortti> I think the platform triplet for it is i386-gnu, even
<heat> yep
<mjg> no, hurd is the shit
<sortie> Considering GNU is meant to be an operating system, THEY FORGOT TO BE AN OPERATING SYSTEM
<brunothedev> just call it gnu
<sortie> shame on them
<mjg> EEY
xenos1984 has joined #osdev
<mjg> does rms even visit gnu.org
<heat> who visits gnu.org?
<heat> I know I haven't
<heat> ever
gog has left #osdev [byee]
<sortie> heat, all the good shit goes down at ftp.gnu.org everyone knows that
<brunothedev> Average gnu/hurd fan vs. Average Sortix enjoyer
<sortie> klange, I love your screenshot collection
<sortie> brunothedev, if you're new to hobby OSes, do check out klange's toaru also. Super impressive work and very inspiring :)
<heat> if you're new to hobby OSes do check out FreeBSD
<sortie> Hey yeah FreeBSD is a nice thing to also check out if one only has tried Linux
<heat> yeah for some definition of nice
<sortie> Like for real, checking out various other ways of doing things is very inspiring
<mjg> the relentless bullying on this channel
<sortie> I am inspired a lot by Linux in my osdev
<heat> it's like a pilgrimage. really makes you appreciate linux
<brunothedev> heat: i would probaply tests some BSDs, but 2 problems: 1. NVIDIA 2. Wayland
<sortie> When Linux makes me sad I make Sortix better
<mjg> :)
<heat> not even BSD devs test BSD, don't worry
<heat> it's just a fake system everyone pretends to use
<brunothedev> heat: FreeBSD
<brunothedev> it seems tha OpenBSD test their os tho
<sortie> FreeBSD is really just macOS developers
<mjg> you got it backwards mate
<heat> shit, could be PayMeToUseItBSD
<brunothedev> sortie: imagine if linus taked the offer to work on macos
<mjg> it is macos users developing freebsd
<mjg> you go to a bsd confrence, you get mac gallore
<sortie> Yeah that's what I meant
<heat> if you go to a Linux convention you get a pentium4 thinkpad gallore
<sortie> I don't think anyone really managed to compile Sortix for real on a mac
<mjg> brah my first laptop was a thinkpad t30
<sortie> Certainly not with ports
<mjg> best thing ever
<nortti> I used a pentium III thinkpad (t20) without a screen like 10 years ago (before the eeepc)
<sortie> mjg, I actually made Sortix-on-Sortix the primary and official way to do it and cross-dev secondary (but well supported)
<sortie> nortti!
<brunothedev> my next gpu wont be nvidia, i cannot even run alpine because they dont compile their drivers against musl
<sortie> :9
<sortie> :)
<nortti> pretty good other than slowness and lack of ram, which, are pretty important
<heat> mjg, exactly, linux people use thinkpads
<brunothedev> sortie: llvm backend?
<sortie> brunothedev, nah modified gcc per https://wiki.osdev.org/OS_Specific_Toolchain
<bslsk05> ​wiki.osdev.org: OS Specific Toolchain - OSDev Wiki
<heat> mjg, fwiw i'm pretty sad you run Ubuntu and not RHEL
<heat> you're definitely a RHEL kind of guy
<brunothedev> want to talk about not running their own os? Have you heard of Plan9 forks?
<sortie> Technically all the Sortix devs use debian
<sortie> heat, what do you run normally btw?
<heat> arch
<heat> 2 out of 3 dentistssortix devs run debian
<brunothedev> sortie: tried debian, too much random bugs
<klange> I honestly do more dev under macOS now...
<sortie> brunothedev, it has stabilized for me now although last year had some annoying bugs
<heat> you should try fedora
<heat> seriously
<heat> good distro
<nortti> klange: can you use docker under macos, or is there another way to build toaru?
<sortie> Personally I just use debian to get a default experience that's good enough so I don't have to worry and tinker
<brunothedev> heat: wanna know what distro i am running ;)
<heat> not particularly but feel free to share
<brunothedev> if i ever get a amd gpu, i will distrohop to alpine linux
ceremony has quit [Remote host closed the connection]
<klange> nortti: I have a native toolchain
<heat> sortie, thing is that I've experimentally verified that the farther you are from HEAD, the buggier it is
<sortie> heat, debian?
<heat> any distro
<sortie> I do run debian testing
<heat> which is why I tend to run bleeding edge only
<brunothedev> heat: lol i am running fedora too
<nortti> honestly what gentoo does with masking seems to be a pretty good approach
<heat> does work pretty well. my workflow has only been broken for a bit a few months back with a linux 6.1 bug
<sortie> I have like zero interest in maintaining a Linux distro just to make things work, when I wanna spend that energy maintaining my Sortix distribution
<mjg> :]
<heat> yeah?
<mjg> fwiw ubuntu works for me almost without a hitch
<heat> i don't do any maintaining here
<brunothedev> anyone have a simple, vga driver in C so i can read, it seems that there are no code on the osdev wiki
<heat> funnily enough the most maintaining I've done on linux systems has been in fucking debian derivatives
<mjg> brunothedev: how much a driver do you want to see
<mjg> as in are you trying to get letters on screen?
<sortie> I got an old desktop with like 8 GB RAM and it OOMs pretty much all the time which Linux is absolutely terrible at and everyone involved should be very ashamed of themselves
<mjg> if so, i have good news
<moon-child> haha
<heat> sortie, works well here
<mjg> sortie: i thought G recently merged some improvements
<mjg> heat: it works like shit on my lap
<sortie> At least the Linux kernel improved a bit or the systemd config did, and Firefox also now supports unloads, now it's just VLC leaking massive amounts of memory
<brunothedev> mjg: literally anything, i can bootstrap a way to make characters and create a console
<heat> yes, new MGLRU stuff
<nortti> sortie: tbh you are running an unsupported setup with no swap
<heat> there you go
<mjg> brunothedev: you just write bytes to a dedicated area and it magicallyshows up
<mjg> brunothedev: i'm sure this is described
<mjg> brunothedev: alternatively just google for a hello world kernel
<sortie> Everyone involved which swap should also be very ashamed of themselves
<heat> sortie, I've found that swapping works pretty well
<sortie> It's super slow on this harddisk and I **never want to swap**. I just want firefox to go nope when I oom and that's it
<heat> hi sortie did you know a corporation on which you have a considerable stock position does a lot of swap work
<brunothedev> mjg: most hello world kernels use text mode i think, i want to draw normally
* sortie . o O (stock swaps)
<heat> sortie, btw what's "nope" here? ENOMEM?
<klange> brunothedev: Are you writing your own bootloader?
<nortti> sortie: the weird thing is that I also run noswap debian unstable with firefoc on 8 gigs and the only time I've had it livelock was with webgl
<sortie> heat, just kill the damn firefox process
<brunothedev> klange: nope
<sortie> Or vlc
<sortie> Or whatever is fucking up
<klange> brunothedev: What bootloader are you using?
<sortie> Just don't blast radius everything else or livelock
<heat> sortie, OOM killer should do that AFAIK
<brunothedev> klange: gonna use grub
<heat> if you've got something big its OOM score will be high
<klange> brunothedev: Tell grub to set up a framebuffer.
<sortie> heat, I know all that, I'm saying, it works poorly. It worked very, very badly. It got better.
<heat> sure
<nortti> heat: if you lack swap, it ends up evicting currently hot code and then loading it off (in sortie's case, spinning rust) disk for a good while before it invokes oom killer
<heat> nortti, yes, because that's better usually
<nortti> aye
<nortti> just saying that sortie is not seeing oom killer trigger even
<heat> and no, should not be "hot code" AFAIK
<heat> AIUI it targets unused page cache stuff
<brunothedev> klange: would that make me denpendant on grub?
<sortie> The problem is reloading off the disk all the time behavior slows performance so much that it never makes any forward progress (perhaps because it's not keeping up) and the system livelocks
<nortti> heat: I've had the entire system lock up for 10 minutes while stuff fills up enough for oom-killer to launch. if that's not evicting hot code I don't know what's going on
<heat> right
<heat> that does sound like swapping out the wrong stuff
<klange> brunothedev: are you using multiboot to load from grub?
<brunothedev> klange: yes
<klange> brunothedev: you're already dependent on grub, then
<heat> i know there are cgroup workarounds for that stuff
<brunothedev> klange: it seems that the specification is offered by grub
<brunothedev> specification != implementation
<nortti> you are unlikely to use any other multiboot bootloader than grub
<klange> 1) Use a preconfigured framebuffer from your boot environment. In grub this means setting "set gfxpayload=keep" (if grub was already running in a graphics mode, otherwise "set gfxpayload=WIDTHxHEIGHTxBPP" how you want it) and getting the framebuffer info from the multiboot data.
<brunothedev> nortti: oh ok
<klange> 2) When you have that working, then write a modesetting driver for the Bochs virtual display adapter implemented by Bochs, QEMU, and VirtualBox.
<klange> 3) Once that's working, write one for VMware's slightly different thing.
<klange> 4) Stop. Any actual graphics hardware beyond this point will take you exponentially longer to write drivers for than the entire rest of your OS.
<heat> nortti, actually multiboot1 is wide-*ish*ly implemented
<klange> heat: there's only one other implementation of multiboot1 in an actual bootloader, and it's an incomplete one in a PXE loader
<heat> syslinux
<nortti> right but do you really want to ship your hobby os with extlinux?
<heat> also qemu
<klange> qemu isn't a bootloader, and also its implementation is horrendously incomplete
<klange> The most complete multiboot1 implementation outside of grub is probably _mine_.
<heat> also implementing a virtio gpu driver is not impossible and probably a decent idea
<heat> at least the modesetting bits are trivial
<brunothedev> gonna try vesa
<heat> no
<heat> use fucking grub
Lumia has joined #osdev
<nortti> brunothedev: you do not want to drop down to real mode and keep around the environment to allow that. your bootloader (= grub) will handle talking with vesa for you
<nortti> also I don't think if you uefi boot you can even use vesa, or can you?
<heat> no you can't
<klange> Absolutely do not do VESA unless you're writing your own bootloader.
<netbsduser> klange: limine also has complete multiboot1 and multiboot2 implementations
<klange> Still? I thought they removed them.
<klange> I think they also called everyone use them idiots, but that might have been a random passerby.
<mrvn> What's mising in qemus multiboot?
<klange> All the graphics stuff.
<brunothedev> soooooo "kernel/drivers/generic/grubfb"?
<netbsduser> it would be slightly unusual to target them if using Limine since there is a protocol which puts you immediately into long mode with a reasonable state, but they're certainly there
<nortti> what's limine?
<netbsduser> nortti: a bootloader which i favour
<mrvn> nortti: a bootloader that already went through 3 new boot protocols because the previous one they designed wasn't good enough.
<nortti> ah
<klange> limine does more than most people's operating systems
<klange> why even bother getting into the hobby at that point
<netbsduser> i thought grub was the one that did that
<netbsduser> i don't even have a zfs driver smh
<klange> grub barely even puts you into a working protected mode
<netbsduser> i count that a weakness
<heat> grub is huge, EFI is huge
<heat> they're all huge
<heat> but limine sucks
<klange> grub itself does a lot, but grub doesn't expose any of that to you
<heat> from that other guy's struggles with the boot protocol I did understand that it's remotely insane
<netbsduser> wrangling with the legacies of x86 is not osdev, it's cbt, and i don't mean cognitive behavioural therapy
<heat> lol
<heat> seriously man, it's not that hard
<heat> and it is part of osdev
<heat> the GDT is all described in the SDM and if you have a decent-ish grasp of segmentation you'll get it pretty quickly, the MMU you'll need to grok sooner or later, then you just toggle a few bits in some registers and there you go, 64-bit
<netbsduser> it's a part that exists only because of the long legacy amd64 sprang from, and a part i am happy to forget about
troseman has joined #osdev
<brunothedev> lets just think about programming with proprietary ARM and device trees for a moment...
<klange> Grub does do too much, but we accepted it because you were going to have it installed anyway to boot Linux.
<brunothedev> grand unified has to mean something eh?
<heat> yes, the "grand" bit really does make sense
<mrvn> heat: truthfully that other guys problems had nothing to do with limine.
<netbsduser> the thinker guy, or whatever he was called?
<mrvn> brunothedev: yeah, lets talk about the device tree. How do I dump it to a file under linux?
<klange> With grub, I can tell people "just use grub until you get things working, then when you feel comfortable you can write your own bootloader" but with Limine it's like... guess you're screwed unless you want to spend a year replicating that functionality!
vdamewood has joined #osdev
<mrvn> klange: what functionality? Other than long mode what does it do more than grub?
<brunothedev> bootloaders are bloat, store the kernel at 512 KiB /s
<bslsk05> ​shikhin/wodscipe - A bootable 512-byte insane IDE. (0 forks/16 stargazers/Unlicense)
<mrvn> brunothedev: 0x4000 for an Rock 5B
<netbsduser> well, there's the exact thing, i have no interest in writing bootloaders (multiboot struck the first blow against that necessity, grub gave the coup de grace, and limine does similarly for the amd64 age)
<mrvn> netbsduser: multiboot should really just add AMD64 as architecture.
<netbsduser> mrvn: yes, they are missing a trick there
<netbsduser> apparently multiboot does support MIPS of all things
<klange> mrvn: it does SMP bringup and has runtime services
<heat> netbsduser, yep the tinker guy
<heat> or thinker or whatever
<klange> Limine has more runtime services than EFI!
<heat> ewwwwwww
<mrvn> klange: oh, haven't seen them yet.
<netbsduser> the only runtime service i know of is the early boot console
<netbsduser> that and the SMP bringup assistant at a push of 'runtime service' definition
<mrvn> SMP bringup is actually something multiboot should have too. It would be really bad to support AMD64 but then for SMP you have to add 16 bit and 32 bit code yourself.
<heat> that's not a push, it's very much a runtime service
<heat> I don't want a runtime assistant for basic OS operation
<heat> not only do you deal with EFI bugs but now you have to deal with amateur bootloader dev bugs? hard skip
<netbsduser> limine deals with the efi bugs instead, which is nice
<netbsduser> they are more accountable than mobo manufacturers as far as bugs go as well
<brunothedev> how to get multiboot_tag_framebuffer, there is no info on the wiki, it seems to be about the multiboot header, on which i set it up on assembly
<klange> brunothedev: Are you using multiboot1 or multiboot2?
<heat> read the spec
<heat> netbsduser, how tf do they deal with the efi bugs?
<heat> that's either impossible or a completely overreach
<heat> complete*
<brunothedev> klange: i used the bare bones page as a reference
<netbsduser> in the sense that those which are relevant to booting should now be worked around by limine, rather than yourself
<klange> What barebones page?
<brunothedev> klange: the c one
<klange> What C one?
<klange> There are _three_ on the osdev wiki.
<heat> netbsduser, yes but how about *everything else*
nyah has quit [Quit: leaving]
<heat> and it's a very small bootloader too
<bslsk05> ​wiki.osdev.org: Bare Bones - OSDev Wiki
<heat> grub is much larger and still has hit multiple problems that cause real regressions in upstream EFI
<klange> brunothedev: That uses multiboot1. There are no "tags". The framebuffer information is at the bottom of the header, and you set one bit flag (0x4) to request it.
<klange> Also that barebones guide is missing most of the multiboot request header...
<brunothedev> klange: can i replace the assembly multiboot header with a c multiboot2 header as seen on the multiboot wiki page without much problem?
<heat> no
<klange> No, they are completely unrelated.
<klange> multiboot2 uses different commands in grub.
cooligans has joined #osdev
<klange> You probably _should_ use multiboot2, but there aren't as many example implementations floating around...
<cooligans> brunothedev: don't use multiboot1
<bslsk05> ​github.com: toaruos/bootstrap.S at master · klange/toaruos · GitHub
<cooligans> I'd rather use the Linux proto rather than multiboot1
<cooligans> that's actually not a bad idea tbh
<heat> naw
<heat> the linux fucking boot protocol?
<cooligans> yes
<cooligans> Linux boot proto > multiboot1
<klange> The Linux boot protocol is not stable and Linux expects things to fuck it up and essentially reboots itself.
<heat> would rather have edward scissor-hands give me a backrub
<cooligans> how is it not stable
<cooligans> it runs on my computer, and it proably runs on yours too
<klange> I would rather pretend I didn't say any of the things I said for the last hour and tell people to use Limine.
<heat> haha
<klange> cooligans: It _changes_ with different kernels.
<klange> It is not a stable _protocol_.
<heat> that's not true klange
<cooligans> well yeah, limine proto beats all, im not that ignorant
<cooligans> the changes are minor
<heat> the linux boot protocol changes, as in "expands"
<cooligans> yeah
<cooligans> but it's backwards compatitble
<heat> but it's still backwards compatible to most relatively-recent bootloaders
<cooligans> yep
<heat> probably not fucking 1998 LILO but oh well
<bslsk05> ​cloudblaze/multiboot2 - multiboot2规范的示例代码 (0 forks/1 stargazers/GPL-3.0)
catern has joined #osdev
xvanc has joined #osdev
<heat> that said, reverse engineering it is probably depressing
<heat> so if I really wanted to I would just take some inspiration from fuchsia, which already did a good bit of that work
cooligans has quit [Remote host closed the connection]