klange changed the topic of #osdev to: Operating System Development || Don't ask to ask---just ask! || For 3+ LoC, use a pastebin (for example https://gist.github.com/) || Stats + Old logs: http://osdev-logs.qzx.com New Logs: https://libera.irclog.whitequark.org/osdev || Visit https://wiki.osdev.org and https://forum.osdev.org || Books: https://wiki.osdev.org/Books
<raggi> Yeah, you could use it as a primitive and then for C runtimes you'd layer your longjmp or whatever to retire context
<raggi> s/retire/restore/
<raggi> for io / concurrency heavy applications, the nice thing is that because the event work is often smaller, scheduling event processors would be more predictable, which moves more fine-grained control back into the scheduler
SpikeHeron has joined #osdev
<zid> *finds yet another article claiming malloc and free are slow*
<zid> It's annoying because it's not like, a wrong statement, but it isn't something you'd ever say unless you're intentionally trying to be misleading imo
<zid> doing something is slower than not doing it, shocker
<zid> "use this language instead, it uses a pool allocator! (which it initializes from malloc)"
<clever> u-boot happens to have a malloc/free that is uber fast!
<clever> malloc increments a pointer by size, and returns the old value, free is a no-op :P
<zid> my bootloader does too it's like 2 instructions
<zid> an add and a mov
<clever> yep
<clever> u-boot uses it in the loader stage, so its kind of a bootloader for the bootloader
<heat> my kernel malloc is pretty fast
<heat> praise be the bonwick and mjg
<heat> and even then it probably suffers from really tiny shitty issues like numa and memory ping-ponging between cores
<zid> I should make mjg write me an allocator too
<heat> he didn't write me the allocator
<heat> unfortunately
<zid> TOO
<heat> three
<heat> tree*
<gog> hi
<heat> gog
<zid> highg og
<zid> gog when is your cavewoman cosplay reveal, Ogg.
<gog> 31 october
<zid> too late for my birthday then
<gog> when is ur bday
<heat> omg my birthday is in like 2 days
<heat> birthday month twiiiiiiiiinssssssssss
<gog> mine is 11 days
<zid> 25th
<zid> heat is getting me a pony
<heat> omgomgomgomgomg
<zid> mjg is getting me an allocator, I just decided.
<heat> do u want my allocator
<heat> i dont think i can ship u a pony
<heat> is good alloc, i promiss
<gog> i can ship you an icelandic horse
<gog> it is not a pony
<heat> is that a unicorn
<kazinsal> it's a mildly tamed reindeer
maxdev has quit [Remote host closed the connection]
dutch has quit [Quit: WeeChat 3.6]
<bslsk05> ​www.youtube.com: John Mulaney-- Horse in the Hospital (Clean) - YouTube
gog has quit [Ping timeout: 265 seconds]
isaacwoods has quit [Quit: WeeChat 3.6]
nyah has quit [Ping timeout: 252 seconds]
<dh`> raggi: synchronous code ends up being continuation passing style, which is a pain in the ass in any language (even scheme)
<dh`> but anyway I don't agree, fault handling doesn't have bounded waits and doesn't require limited resources, unless you are (a) not supporting mmap so you never end up in a file system, and _also_ (b) swapping to a very restricted set of simpleminded I/O devices
<dh`> maybe you don't allow swapping on umass devices, but sata is plenty complicated enough that you don't want to try doing it in asm with a limited set of registers or anything
<dh`> plus you still get huge read latencies
<heat> I'm confused
<heat> posix pselect says "If sigmask is not a null pointer, then the pselect() function shall replace the signal mask of the caller by the set of signals pointed to by sigmask before examining the descriptors, and shall restore the signal mask of the calling thread before returning."
<heat> this sounds like it replaces; do_select(); reset_sigmask();
<heat> right?
<heat> but this doesn't seem to be the behavior IRL
<heat> for instance
<bslsk05> ​grok.dragonflybsd.org: sys_sig.c (revision da22d5f8) - OpenGrok cross reference for /netbsd/sys/kern/sys_sig.c
<heat> l_sigmask restoration depends on if (sigispending(l, 0)) {
terrorjack has quit [Quit: The Lounge - https://thelounge.chat]
terrorjack has joined #osdev
<heat> ahhhhh
<heat> sigsuspend and pselect, ppoll have this quirk where the old sigmask should only be restored at sigreturn time
<heat> I was doing replace_sigmask(); do_select(); reset_sigmask();. the old sigmask had the relevant signal masked, so it returned -EINTR, GNU make looked at EINTR and circled back to pselect, with the signal still pending because it was masked
carbonfiber has quit [Quit: Connection closed for inactivity]
rurtty has quit [Ping timeout: 268 seconds]
[_] has joined #osdev
[itchyjunk] has quit [Ping timeout: 265 seconds]
<zid> ⍼ heat I bought you a new emoji
<heat> that's beautiful
<heat> thanks <333
<heat> 🩤 here's some unicode for you too
<raggi> dh`: it sounds like you're applying the concept to Linux, or at least a bsd/unix type OS. I'd recommend reading the paper I linked. Some things can be built differently!
<heat> im like 1 or 2 steps away from being able to build my base OS under itself
<heat> 💯
<dh`> raggi: I'm applying the concept to a "real" OS that has stuff like VM and SATA
<dh`> if you have something very small and embedded, it likely doesn't have paging, sure.
<dh`> or memory protection either, or any of the other things that make blocking expensive
<dh`> the abstract of the thing you linked reads like a buzzword bingo
<dh`> I guess it's some kind of supercomputer thing?
[_] has quit [Remote host closed the connection]
<dh`> tbh, sounds like it probably comes from one of those people who thinks the OS is only there to get in the way and then reinvents it badly
<dh`> oh well
heat has quit [Ping timeout: 260 seconds]
GeDaMo has joined #osdev
Affliction has quit [Remote host closed the connection]
Affliction has joined #osdev
zaquest has quit [Remote host closed the connection]
wootehfoot has joined #osdev
wootehfoot has quit [Ping timeout: 265 seconds]
zaquest has joined #osdev
nyah has joined #osdev
<raggi> I'm not going to engage anymore in this topic with you. I don't think you're engaging in good faith anymore
<mrvn> Givejn todays ram sizes and the speed difference between ram and block devices I'm perfectly fine with no swapping and no mmap.
<mrvn> With mmap you prsctically assume there will never be an IO error and never a crash.
roan has quit [Quit: Lost terminal]
Affliction has quit [Quit: Read error: Connection reset by beer]
Affliction has joined #osdev
<dh`> well ok then
<dh`> mrvn: quite true but mmap for executables remains useful for sharing
<dh`> however, I'm not super inclined to hang around to be insulted
dh` has left #osdev [#osdev]
SpikeHeron has quit [Quit: WeeChat 3.6]
<mrvn> The current design has a per-page struct for physical pages of 64bit. It's either allocated to one process by pointing to the owning process, 0 for free or if the lowest bit(s) is set it's a ref-count for a shared page or something I haven't implemented yet. Or maybe a pointer to a more complex VM struct. So far I want to limit this to read-only sharing excatly for executables and the like. But while there
<mrvn> are many nice things you can do with sharing it al turns into a complex mess and I want to remain KISS.
<mrvn> Hehe, I just ordered food to be delivered at 14:45 (earliest they do) and liferando promises me it will be here in 50 minutes.
<mjg> what mmap is good for is avoiding double copies
<mrvn> how do you mean double?
<mjg> imagine a multi GB file which you do have cached and multiple processes which want to access it RO
<mrvn> twice or duplicate?
<mjg> any tlb wins aside from mmap, the aboe is just such a memory saving opportunity
<mrvn> Aeh, how would you sabe TLB entries? Global mappings would allow every process to access the page and non-global mappings get invalidated on every context switch and fault. Or with ASID you get multiple TLB entries.
<mrvn> Which is actually what I might implement: globally shared pages. You still have to get/put the page but if you can guess the address you can just access it.
<mrvn> Which would be fine for any world readable file.
pretty_dumm_guy has joined #osdev
<mrvn> mjg: Even COW gets horrible complex. At first glance it's simple: map the page read-only into the process and when it faults on write make a copy. But then you have to change the original owner of the page to read-only and COW them too. And you have to ref-count how many people have access to the original page and if the count becomes 1 drop the COW bit.
<mrvn> Which means invalidating pages in a nother process that might be running on another core.
dutch has joined #osdev
rurtty has joined #osdev
gog has joined #osdev
jimbzy has quit [Ping timeout: 246 seconds]
vdamewood has joined #osdev
vdamewood has quit [Read error: Connection reset by peer]
vdamewood has joined #osdev
selve has joined #osdev
vdamewood has quit [Read error: Connection reset by peer]
vdamewood has joined #osdev
aleamb has quit [Quit: WeeChat 2.8]
[itchyjunk] has joined #osdev
isaacwoods has joined #osdev
energizer has quit [Quit: ZNC 1.7.0+deb0+xenial1 - https://znc.in]
energizer has joined #osdev
jimbzy has joined #osdev
<bslsk05> ​'c8sp4ueazqs91' by [idk] (--:--:--)
ay is now known as Andrew
epony has quit [Remote host closed the connection]
<mjg> heat your slab alloc is slow as fuck, worse than solaris
<mrvn> is that using free lists or bitmaps?
<mrvn> is it bouncing a lock between cores?
<mjg> i'm gonna roast him when he shows up
<mjg> there is false sharing and atomics when there should be none
selve has quit [Remote host closed the connection]
selve has joined #osdev
xenos1984 has quit [Ping timeout: 260 seconds]
xenos1984 has joined #osdev
heat has joined #osdev
<heat> mjg, no its not
<heat> i need evidence
<heat> I don't have atomics in my slab allocator
<heat> well, I do, but they're relaxed
<heat> they're there just so the compiler doesn't fuck it up
<heat> mjg, do you want a cool flamegraph? https://gist.github.com/heatd/eaaf5914bc3efe4868988426b712b047
<bslsk05> ​gist.github.com: onyx-makevmonyx-j8.svg · GitHub
<heat> I redacted sched_idle because it's still ridiculously dominant
<heat> i'll finally get to reworking vm and fs, block io
<heat> it's very fun how cheap vfork is compared to fork
<heat> and do note that this is still on tmpfs
<heat> ext2 should be fun
<mjg> first, atomics are always more expensive than non-atomics, regardless of what barriers you slap on them
<mjg> and it so happens all atomics on amd64 are de facto coming with a full barrier
<heat> why?
<mjg> ask intel
<mjg> this is what ultimately happens
<mjg> so
<mjg> pcpu->touched.add_fetch(1, mem_order::relaxed);
<mjg> it is an invariant that touched is either 0 or 1
<heat> I see
<heat> .store?
<mjg> what you want to do here is atomic_store
<heat> store should be a mov
<mjg> yes, no *locked* ops
<mjg> but here is a little subtelnty
<mjg> you need a release barrier
<mjg> erm
<Ermine> Wow, this flamegraph is interactive
<mjg> a compiler barrier
<heat> Ermine, they all are, it's cray cray
<mjg> *after* you 0 -> 1 and before you 1 -> 0
<mjg> otherwise, in priciple, the compiler can reshuffle it
<mjg> also sched_disable_preempt and sched_enable_preempt should bei nlines
<heat> but they're atomics though? I thought the compiler couldn't reorder atomics
<mjg> you literally told it mem_order::relaxed
<heat> so acquire and release?
<mjg> you would only need acq and rel if there were other cpus at play
<mjg> here you only need to synchro the code against an ipi handler running on the same cpu
<mjg> and to that end you only need the code emitted in right spots, hence compiler barriers
<heat> i'll keep the acq and release here for the time being
<heat> it has no codegen in x86 for stores
<heat> fuck other architectures am I right
xenos1984 has quit [Ping timeout: 250 seconds]
<mjg> i'm not signing off on this shit
<mjg> !
<mjg> struct slab_cache_percpu_context
<mjg> { void *magazine[SLAB_CACHE_PERCPU_MAGAZINE_SIZE]; int size; atomic<int> touched;
<mjg> };
<mjg> i already told you this false-shares against other cpus
<heat> Signed-off-by: Mateusz Guzik <mjg@freebsd.org>
<heat> done
<mjg> you need to align this
<heat> ok dad
<zid> Every time I call mjg daddy he still doesn't buy me things, btw
<zid> (good sentence zid, 10/10)
<heat> father
<heat> i have done what you asked
<zid> now he sounds like clergy, this is difficult
<heat> catholic moment?
<mjg> heat i'm your father
<heat> noooooooooooooooooooooooooooooooooooooooo
<mjg> you will unlock multicore performance if you join the dark side
dutch has quit [Quit: WeeChat 3.6]
<heat> you mean R C
<heat> ew
<Ermine> os wars
<zid> "No heat, what I am trying to tell you is that when you are ready, I am your father"
<zid> I think I mixed my movie quotes
<heat> no os wars
<heat> os hugs
<zid> heat's always hugging solaris afterall
<heat> mjg, anyway i've been meaning to tell you that this is literally the lowest of the hanging fruits
<heat> we optimized the shit out of a few testcases, yes
<Ermine> os love
<heat> have you looked at my pipe.cpp? it's hilarious
<heat> i just allocate a big buffer and memmove on read
<heat> my vm subsystem is very tiny allocation happy
<mjg> ok, mofo, here is some real talk before i have to go afk
<heat> i literally block for like 3/4ths of the compile time
<heat> on -j8
<mjg> it is crucial to have core routines performance-sanitized as otherwise they can contribute to weird anomalies
selve has quit [Remote host closed the connection]
xenos1984 has joined #osdev
<mjg> microoptimizng the shit out of them is not necessarily warranted, but something like trivially avoidable branches or atomics is definitely "rip it out" material
selve has joined #osdev
<mjg> i don't know why people are notorious for writing them in an incredibly sloppy manner
<heat> because it's hard
<mjg> not having atomics in slab alloc is not hard man
<zid> mjg when are you writing me an allocator
<zid> it doesn't even have to be smp capable
<mjg> one week before openbsd starts scaling
<zid> not even for my birthday?
<mjg> i don't make the rules
<heat> wdym openbsd starts scaling
<zid> My desktop has obscene anti-scaling now that it has a 12 thread cpu, I still only use 1 of them
<zid> so now I am 8% efficient instead of 12.5%
<heat> why
<zid> what could I use more for, I don't need to run 12 copies of nethack
<zid> or dwarf fortress
<heat> you can though
<zid> I could, but I am not going to ever need to
<mjg> heat: add openbsd binary compat and build the system faster than they do
<mjg> kthx
<heat> nah
<mjg> now that you mention pipes, i had seen a pipe bench somewhere
<zid> The only bench that matters is `ab` anyway, everybody knows that
<zid> if it isn't WEB it isn't real
<mjg> ah ab
<mjg> did they fix it?
<mjg> heat: https://dpaste.com/DSX6N7TFX that will be $300 consulting fee
<bslsk05> ​dpaste.com <no title>
<heat> LMAO
<mjg> albeit i have to note i tis quite unclear if zero page there is a good idea to begin with
Ermine has quit [Quit: Cya!]
vdamewood has quit [Quit: Life beckons]
Ermine has joined #osdev
Ermine has quit [Quit: Cya!]
smach has joined #osdev
Ermine has joined #osdev
epony has joined #osdev
wootehfoot has joined #osdev
<Clockface> about when did most computers start using SATA instead of ATA?
<Clockface> better question, NVME vs SATA
<heat> most computers still use SATA
<heat> new computers use NVMe though
<Clockface> alright
<heat> probably around 2-3 years ago
<Clockface> and ATA was succeeded through the 2000's right?
m3a has joined #osdev
<heat> i believe so
dutch has joined #osdev
<mxshift> ATA was definitely still common in the late 90s and early 00s. I recall SATA being somewhat novel in 2002-2003
<zid> I didn't get my first sata drive until 2010
<bslsk05> ​'Western Digital 1TB Black SATA 6.0Gb/s SATA3 Hard Drive Unboxing & First Look Linus Tech Tips' by Linus Tech Tips (00:02:26)
<heat> omggg old linus
<mjg> i like that linus takes breaks from the kernel to do hdd unboxing
<mjg> stand up guy despite what people would have you blieve
<heat> i think its the opposite
<heat> he takes breaks from hdd unboxings to do kernel work
<heat> that's why he's so cranky sometimes
<mjg> right
<mjg> touch choices every day
<mjg> unbox this drive proparly damaged in transport or shit on people on lkml
<mjg> some times you have to do both
<mjg> full time job
<heat> you also choose between shitting on freebsd devs or shitting on the lkml
<heat> it's quite a choice really
GeDaMo has quit [Quit: Physics -> Chemistry -> Biology -> Intelligence -> ???]
roan has joined #osdev
Ali_A has joined #osdev
dutch has quit [Quit: WeeChat 3.6]
rurtty has quit [Ping timeout: 268 seconds]
ss4 has joined #osdev
wootehfoot has quit [Ping timeout: 265 seconds]
dutch has joined #osdev
scoobydoo has quit [Ping timeout: 265 seconds]
scoobydoo has joined #osdev
ss4 has quit [Read error: Connection reset by peer]
roan has quit [Quit: Lost terminal]
[itchyjunk] has quit [Remote host closed the connection]
* geist yawns
<heat> sup geist
<geist> woke up late. still taking cold meds, been sleeping for like 12 hours
Ali_A has quit [Quit: Client closed]
sbalmos has quit [Quit: WeeChat 3.6]
sbalmos has joined #osdev
carbonfiber has joined #osdev
xenos1984 has quit [Read error: Connection reset by peer]
xenos1984 has joined #osdev
<zid> no yawning, mr contageous disease
\Test_User has quit [Quit: e]
smach has quit [Remote host closed the connection]
pretty_dumm_guy has quit [Quit: WeeChat 3.5]
wgrant has quit [Quit: WeeChat 3.5]
wgrant has joined #osdev
* moon-child coughs aggressively
dutch has quit [Quit: WeeChat 3.6]
wgrant has quit [Quit: WeeChat 3.5]
nyah has quit [Ping timeout: 268 seconds]
wgrant has joined #osdev