klange changed the topic of #osdev to: Operating System Development || Don't ask to ask---just ask! || For 3+ LoC, use a pastebin (for example https://gist.github.com/) || Stats + Old logs: http://osdev-logs.qzx.com New Logs: https://libera.irclog.whitequark.org/osdev || Visit https://wiki.osdev.org and https://forum.osdev.org || Books: https://wiki.osdev.org/Books
<clever> epony: and the wiki page claims it was fixed in a software update...
<demindiro> I'm putting my money on Apple having some custom extension that tries to detect Apple(tm) hubs and it never got tested with non-Apple hubs
<clever> demindiro: usb-c does have a fairly complex protocol, seperate from usb
<epony> secrecy mist sets in foggily ;-) unfortunately the other security flaws are HW related (unfixable) and are present on many ARM CPUs as well (at least the speculative execution flaws resulting in data leaks out (crypto and other privacy) off the CPU cache at a rate of streaming audio 50 KB/s)
SGautam has quit [Quit: Connection closed for inactivity]
<Iris_Persephone> clever: This is a pure guess, but it might have been due to the fact that M1 chips are used on both Macs and tablets?
smach has joined #osdev
xenos1984 has quit [Read error: Connection reset by peer]
<Iris_Persephone> I don't think that the tablets have more than one USB-C port
<heat> mjg, I'm 4x faster than netbsd on open3_processes -t 4
<heat> are you proud dad
<mjg> hm
<mjg> open3 is doing an O_RDWR call, which isk ind of stupid
<mjg> you don't get these in parallel on one file in real workloads
<mjg> O_RDONLY would be a different story
<mjg> that said
<mjg> are you sure your code is correct there?
<heat> correct where?
<mjg> notably if there are write accessed granted, execve of the file must fial
<mjg> to that end kernels track writers there
<mjg> and i suspect netbsd is doing it in a highly pessimal manner for historical reasons
<mjg> while you possibly don't do it at all
<heat> I don't do that
<mjg> right
<heat> linux doesn't either
<heat> at least not by default
<mjg> i guarantee linux oes it
<heat> you actively need MAP_DENYWRITE
<mjg> that's different
<mjg> i guarantee if you keep a file open for writing, doing an execve on it will fail with ETXTBSY
<heat> you're talking about ETXTBSY right?
<mjg> i had an openro* variant osmewhere
<mjg> where tf is it
<dh`> mjg: that is not what ETXTBSY is about
<mjg> this is what you need to get should you try to execve the file in this case
<mjg> heat: can you get a flamegraph form netbsd? theyh ave dtrace
<heat> RDWR and RDONLY have no perf difference
<heat> just tested it
<heat> yeah sure
<mjg> dtrace -x stackframes=100 -n 'profile-997 /arg0/ { @[stack()] = count(); } tick-60s { exit(0); }' -o out.kern_stacks
<mjg> then the standard treatment
<mjg> weishit, i lost my openro* tests
<mjg> should have made a PR at the time
<mjg> heat: you can ^C dtrace after running the bench for few s
<mjg> make sure to rm the file if you re-start
<heat> just a sec
<heat> setting up sudo
<mjg> also what kernel is this
<dh`> ETXTBSY is a hack to deal with the limitations of MAP_PRIVATE in mmap()
<mjg> there has been a number of performance improvements made about 2 years ago
<heat> the biggest datapoint I got from setting up all BSDs is that all your shells fucking suck
<dh`> and it doesn't actually work, you can write to shared libs
<heat> mjg, 9.3 generic
<mjg> and it is very plausible you are not running them if this is a 9 kernel
<mjg> ye i don't htink you have the speed ups
<heat> dtrace: invalid probe specifier profile-997 /arg0/ { @[stack()] = count(); } tick-60s { exit(0); }: probe description :::profile-997 does not match any probes
<heat> how do I get the new kernels?
<mjg> i'm not debugging this :>
<mjg> dh`: ^^ ?
<mjg> i'm guessing there are up to date snapshots somewehere
<mjg> but then you will need to recomile the kernel to whack debug opts
<heat> i dont mind building it
<mjg> now that's dedication
<heat> damn right
<mjg> i applaud your lack of lie
<mjg> life
<heat> 💯
<heat> it's 1am dude
<heat> what am I supposed to do? sleep? lmao
<mjg> approved
<Iris_Persephone> You're European?
<mjg> look man just take a look around the download website
<bslsk05> ​nycdn.netbsd.org: Index of pub/NetBSD-daily/HEAD/202210030500Z/amd64/installation/cdrom/
<mjg> there you go motherfucker
<dh`> you can get kernels from the build snapshots, they're in NetBSD-daily/
<mjg> he will still need to recompile them
<mjg> is it legit to get the source, just build and install kernels
<mjg> and boot them on the 9.3 branch?
<bslsk05> ​nycdn.netbsd.org: 404 Not Found
<mjg> top of dev branch, whatever you name it
<dh`> or one copy of that at a time
* dh` glares at his mouse
<mjg> but they have debug, so not fit for benchmarking
<mjg> heat: ok mofo do this: git clone https://github.com/NetBSD/src.git
<dh`> what, not cvs? :-p
<dh`> anyway yes you can build a current kernel and boot it with a -9 userland
<mjg> heat: ./build.sh -m amd64 -j BIGNUMFUCKER -u tools
<heat> chill
<mjg> then you find and copy GENERIC, whack DIAGNOSTIC and whatever other debug
<mjg> heat: ./build.sh -m amd64 -j BIGNUMFUCKER -u kernel=MYCONF
<heat> what's MYCONF
<dh`> GENERIC
<mjg> cp GENERIC MYCONF
<dh`> unless you want a custom kernel
<mjg> nano MYCONF # for you i presume nano!
<heat> is that an insult
<mjg> he needs to whack debug opts and preferably not edit generic
<heat> I use ed, the standard text editor
<mjg> or perhaps you have a variant which does not have them
<mjg> heat: my bad!
<mjg> anyway build.sh will tell you how to install the kernel
xenos1984 has joined #osdev
<dh`> hmm
<dh`> GENERIC:makeoptions COPTS="-O2 -fno-omit-frame-pointer"
<dh`> I guess for hardcore benchmarking you'd want to change that
<dh`> but it's not unreasonable to test GENERIC, you know
<dh`> it's what people will generally be using
<heat> i'll want to remove fno-omit-frame-pointer though
<wxwisiasdf> dammit
<mjg> unless something(tm) changed you got extra atomics stemming from diagnostic et al
<mjg> and extra reads on bouncing cache ilnes
<mjg> heat: anyway what's the number you get on 9.3?
<mjg> heat: -t 1 and -t 4
<heat> oh wait
<heat> sorry, that was stupid
<heat> I do want -fno-omit
<dh`> if you find that DIAGNOSTIC makes a significant performance difference, we want to know about it
<heat> mjg, 570K t 4
<dh`> it is not supposed to
<heat> 389K t1
<mjg> heat: you should keep a sheet somewehre with these results
<mjg> in an organized form
<heat> why
<mjg> dh`: unless you whacked the anti idiom, you have routines which conceptually do: unref(.. obj) { KASSERT(obj->ref > 0); if (atomic_sub_int(obj->ref, 1) == 0) ... }
<mjg> this is pessimal
<heat> is pessimal your favourite word
<dh`> malissimal
<mjg> heat: dude you if you need an explanation why data needs to be organized i have bad news
<heat> why do I want to keep it
<mjg> for future reference, for one
<mjg> wanna claim you are faster than X? you need to be able to at least how the numberZZZ
<mjg> show
<dh`> traditionally you write stuff like that in your lab notebook, and lab notebooks are numbered and tracked so they don't get lost
<mjg> what version they got obtained from
<mjg> andw hat, if anything, you changed to the runtime
<mjg> now that's openbsd-smp-esque bad english, but the point should be clear
<heat> lmao
<heat> im waiting for my toolz to build
<mjg> it will take quite some time
gog has quit [Ping timeout: 250 seconds]
<heat> really?
<heat> I was assuming I just had binutils and gcc left
<dh`> gcc isn't exactly super fast to build
<mjg> i do find it plausible it is faster to reinstall
<mjg> from fresh snap
<mjg> and then build the kernel
<heat> aw shoot
<mjg> heat: btw are you testing it against tmpfs?
<heat> if I interrupt build.sh tools will it start over?
<heat> mjg, yes
<mjg> not if you add -u i think
<mjg> just check the help
<mjg> they have something to make it incremental
<heat> so that was a lie
<mjg> what wsa
<mjg> tools finished?
<heat> it's rebuilding
<heat> everything
<heat> I forgot to add -j so I noticed on htop it was only using a single thread
<mjg> -u Set MKUPDATE=yes; do not run "make cleandir" first.
<dh`> -u for update, yeah
<mjg> Without this, everything is rebuilt, including the tools.
<heat> ....i may have forgotten adding that
<mjg> who tf knows, maybe it will restart each time
<heat> wtf
<heat> not even the mighty BSD can escape the clutches of GNU
<heat> lots of configure scripts
<heat> lovely
<dh`> also -o to skip the make-the-objdirs run
<dh`> the tools are supposed to be able to build on solaris, there's not a lot of choice
<mjg> heat: i'm about to check out for the day
<mjg> heat: should you get to a position to bench, don't claim any victory unless you get a flamegraph to validate the result kthx
<heat> are you sure you want to build on solaris
<dh`> no
<heat> well, there you have it
<mjg> dh`: what's netbsd command to load kernel modules
<dh`> or rather, I am sure *I* don't want to build on solaris
<mjg> lkm something?
<dh`> modload I think
<dh`> I forget, I don't allow modules in my kernels
<mjg> and where are modules normallyk ept?
<dh`> in /stand
<mjg> heat: find /stand | grep profile
<heat> aha
<heat> it's a module?
<mjg> it is defo a module and maybe it fails to autoload
<mjg> so ^^^
<heat> kk
<mjg> if you can load it now it would be nice
<mjg> then back to dtrace -x stackframes=100 -n 'profile-997 /arg0/ { @[stack()] = count(); } tick-60s { exit(0); }' -o out.kern_stacks
<mjg> even from the build just to make sure it all works
<mjg> at least on 9
<heat> i'm building on linux
<mjg> :)
<heat> cry bsd man
<mjg> hopefully there were no changes which will prevent 9 dtrace from working on 10
<mjg> then ^Z it
<mjg> and try to get the fg from open3 -t 4
<heat> yeah it works
<mjg> o/
<heat> let's take a look
<bslsk05> ​gist.github.com: nbsd-open3-4.svg · GitHub
<heat> mjg, dh`
<mjg> can you please benchmarking using this going forward
<mjg> ouch
<heat> already did
sprock has quit [Ping timeout: 265 seconds]
<dh`> for modules you need to use a module that matches the kernel version
<mjg> well even if you flipped the flag locally, can you please use the openro3_processes binary :>
<mjg> will avoid possible future confusion
<mjg> anyway the bottleneck here is the old bsd idiocy
<mjg> which i did fix in freebsd fwiw :)
<heat> what is it
<mjg> mount point traversal calls into a fs-specific routine to grab the root vnode
<mjg> which normally is implemented by going to into the bowels of the fs
<mjg> trying to find it from scratch
<mjg> that's what the code is serializing on
<heat> I can see a blocking vcache_get
<mjg> that's part of it
<mjg> i avoided the problem by adding struct vnode *mnt_rootvnode;
<mjg> to struct mount
<heat> my kernel does mounts direcly on top of dentries
<heat> but then again, it does require them to stay alive contrary to BSD apparently
<mjg> i'm not going to rant about this bit
<mjg> there is a lot of badness here which adds tons of cmplexity and reduces performance for no gain
<dh`> pinning the volume root vnode in the vnode cache seems like a reasonable tradeoff
<mjg> right, it';s like one per mount point and you keep hitting it
<heat> why are you using a mutex there?
<mjg> netbsd
<heat> it seems to be spinning but it's a mutex nontheless
<dh`> netbsd has adaptive mutexes
<mjg> i don't know the granularity there, possibly one lock for the entire cache
<dh`> (someone thought this was a good idea)
<mjg> this is a good idea, but the lookup should not be mutex protected in this case
<dh`> the vnode cache is more parallel than that but I don't know the details
<dh`> it got reworked a few years back
<heat> what's this Xspllower I keep seeing?
<dh`> am I reading this thing correctly? it's spending most of its time locking in the vnode cache and locking vnodes?
<mjg> return from interrupt handling
<dh`> as well as getting a lot of time billed in spllower() which has to be bs
<heat> yep
<mjg> dh`: spllowr is a dtrace limitatin
[itchyjunk] has joined #osdev
<mjg> it does not track interrupts
<mjg> so whatever happens there is accounted to the spot where they get reenabled
<mjg> hence the routine
<dh`> what does the horizontal ordering mean?
<heat> nothing
<dh`> (I've seen these plots before of course but I'm not sure I've ever looked at one that closely)
<heat> there's a mode you can enable where it stops collapsing stacks and does a timeline there
<heat> but this isn't it
<mjg> left to right is alphabetical afair
<mjg> colors are random
<heat> why do you grab an fs lock on close?
<mjg> htat's part of the vfs bullshit
<mjg> ->v_data is your filesystem-specific stuff, right
<heat> sure
<mjg> at any point when the vnode is not locked it can get "doomed" (for example forced unmount)
<mjg> that is going to whack some of it, including ->v_data
<mjg> so in order to safely access it you take the lock
<mjg> in fact it is the layer taking it for you
<mjg> on top of if the aforementioned writers accounting probably depends on it in nbsd as well
<mjg> freebsd also takes the lock there, but i can remove it over the weekend
<dh`> VOP_CLOSE does not actually do much
<mjg> all ingredients but 1 are there
<heat> I built the new kernel
<mjg> dh`: no, that's part of my complaint about the fuckng vfs layer
<heat> where are the modules?
<mjg> i think modules have a separate target
<mjg> 'modules' it seems
<dh`> tmpfs_close does nothing at all in fact
<mjg> then you installmodules DESTDIR=/tmp/lolo
<mjg> dh`: .. but the fucking layer takes the lock anyway. <-- my complaint
<dh`> yeah
<mjg> dh`: it needs to be relegated to the filesystem
<dh`> if the locking happens above the vfs layer, it has to be conservative
<mjg> VOP_UCLOSE
<dh`> if it happens below, every fs gets the chance to do it wrong and reinvent the same stupids over and over
<mjg> if the fs can deal with it, great, otherwise it gets the current treatment
<dh`> given those choices I prefer "above"
<heat> can't you just if (vnode->close) { VOP_LOCK(); vnode->close(); VOP_UNLOCK();} or whatever
<mjg> what
<heat> i dont know
<heat> this is just bsd pseudocode
<dh`> yes and no
<mjg> i think i see what you mean, but no
<mjg> see the writer adjustment note for example
<dh`> that's what linux does, but it ends up oogly in a lot of cases
<dh`> I could imagine a scheme where part of the vnode ops table declares the fs's locking requirements of each of the ops
<heat> sure, but it works
<mjg> for the life of me i can't remember if ad@ did sometning about mp traversal
<mjg> we will ind out soon enough i guess
<dh`> like the "this fs sucks and requires the kernel biglock" flag, but more detailed
<dh`> but without some kind of way to crosscheck/enforce it, it'll never be right
<heat> start off conservatively and set it for every filesystem?
<dh`> a year or two ago ad dumped some namecache fast-forward code into namei
<dh`> well, the problem is that things change over time
<mjg> heat: are you in positin to test the new kenrel or not
<dh`> and so you really need some kind of minimal crosscheck of whether the locking declaration is correct
<heat> mjg, iz building modules
<heat> i can probably boot and see what results I get
<heat> actually
<heat> I got the modules done just now
<mjg> maybe not while building modules man?
<mjg> :]
<mjg> oh ok
<dh`> the ffs close op syncs up the timestamp handling, which is necessary and requires a write lock
<mjg> only if there is a timestamp change to begin with, which it very well may not be
<dh`> ...which can race
<wxwisiasdf> does anyone know of native cobol compilers?
<mjg> but that would already race
<heat> wxwisiasdf, cursed ass question
<mjg> wxwisiasdf: are you the guy from this tweet https://i.imgur.com/H1lkYxC.png ?
sprock has joined #osdev
<dh`> no, if you are holding the vnode lock a setattr can't get in at the same time
<dh`> if you don't, it might
<wxwisiasdf> what no
<heat> bah
<mjg> wxwisiasdf: then wtf man
<heat> the installation fails
<mjg> how
<heat> I'll try to install the modules anyway
<heat> x86_64--netbsd-install: /home/pfalcato/modules/usr/libdata/ldscripts/kmodule.inst.KvMh8l: mkstemp: No such file or directory
<heat> I think the actual modules are installed though
<heat> lets see
<wxwisiasdf> mjg: i just like cobol
<wxwisiasdf> there is something truly charming about it
<heat> seems to boot
<heat> are you fucking joking
<heat> cobol has a charm??
<mjg> wxwisiasdf: i'm giong to unsee your response. best of luck :)
<wxwisiasdf> :/
<wxwisiasdf> why nobody like cobol for osdev
<wxwisiasdf> cobol is as capable as c to deliver a fully fledged kernel ;-;
<dh`> uh......
<kof123> is there an example (that is kernel you know for sure was written in cobol, some ancient proprietary thing?) track down what they used...
<kof123> *that is, a
<wxwisiasdf> i am writing a kernel in cobol as we speak
<heat> mjg, i can't get a fucking flamegraph rn
<heat> but
<heat> 1.1M t=4
<dh`> heat: that failure looks like trying to install the whole system without first building it
<heat> 418k t=1
<dh`> but also, you don't need to install modules to use them if you load them by hand
<heat> wait, i can get a flamegraph
<mjg> so about twice the speed at -t 4
<mjg> which is still low
<mjg> well let's see what you got there
<heat> ok
<heat> dtrace: invalid probe specifier profile-997 /arg0/ { @[stack()] = count(); } tick-60s { exit(0); }: "/usr/lib/dtrace/psinfo.d", line 46: failed to copy type of 'pr_arglen': Type information is in parent and unavailable
<heat> oopsie
<mjg> no bggie
<mjg> rm /usr/lib/dtrace/psinfo.d
<mjg> keep removing until it starts up
<mjg> getting stacktraces does not need to know squat about types, so it will be fine for this purpose
<heat> ah yes
<heat> looks a lot nicer
<bslsk05> ​gist.github.com: nbsd2-open3-4.svg · GitHub
<heat> there's still a big uncomfortable lock in tryemulroot though
wxwisiasdf has quit [Quit: leaving]
<heat> dh`, do you not use the x86 TSC for timing?
wxwisiasdf has joined #osdev
<mjg> i suspect that's the terminal vnode lock
<bslsk05> ​gist.github.com: nbsd2-vfsmix-t4.svg · GitHub
<heat> there's a good chunk of cpu being burned in inl instructions
<mjg> heat: sysctl -w vfs.timestamp_precision=1
<mjg> defualt is to use nanoseconds, which do rdtscp
<mjg> and that kills perf
<heat> dude, this is acpi timer
<heat> it's way worse
<heat> not a thing btw
<heat> that sysctl
<mjg> wait
<mjg> cna you sysctl -a | grep timestamp
<heat> ah
<heat> i see
<heat> vfs.generic.timestamp_precision
<mjg> lol
<mjg> welp that was a freebsd name, i know they copied the code
<mjg> assumed the name sruvived
<mjg> heat: can you bench open1 please
<mjg> that's *separate* files
<dh`> timing uses whatever timecounters the system probes and are enabled
<mjg> the q is if the vnode lock problem disappears
<heat> I have less inl samples now
<dh`> try sysctl kern.timecounter.hardware
<heat> doubles to 20k vfsmix
<dh`> and kern.timecounter.choice
<heat> maybe I'm screwing over netbsd on virtualbox
<heat> I don't know if they enable the tsc shizzle that makes it usable
<mjg> ye i wuld not bench this one right now
<mjg> just get open1
<mjg> -t 1 vs - t 4
<heat> also patching for O_RDONLY
<heat> 1.5M t4 vs 416K t1
vdamewood has joined #osdev
<dh`> timing on VMs is dodgy enough that it's all dubious anyway
<heat> i should run this on qemu or something
<heat> I think KVM does a better job with TSC stuff
<heat> particularly, I don't think they vmexit on those
<dh`> dunno
<heat> maybe I'm wrong but top-of-the-profile acpi timer inl kind of suggest to me that it's because of vmexits
<dh`> I wrote sys161 so it specifically tracks virtual time properly, but the timing models it uses are trash
<dh`> (particularly for any kind of parallel scaling)
<dh`> should really do something about that eventually. in my copious spare time
<mjg> heat: so what's the fg on that one
eck has quit [Quit: PIRCH98:WIN 95/98/WIN NT:1.0 (build 1.0.1.1190)]
<mjg> heat: there clearly is a bump
<bslsk05> ​gist.github.com: nbsd2-open1-4.svg · GitHub
<mjg> ye
<mjg> so open(..., O_RDONLY) takes an exclusive lock on t
<mjg> on it
<mjg> aka write lock
<mjg> your kernel does not suffer this problem i presume
Iris_Persephone_ has joined #osdev
<mjg> so as is i would call it a legitimate win for these 2 benchen
<heat> win for who?
<mjg> onyx
<heat> hell yea
<dh`> whether the open is readonly has very little effect on whether the internal operations need to write things to memory
<heat> netbsd doens't seem to do magazine fuckery
<heat> in the slab allocator
<mjg> dh`: open for reading should scale man
<dh`> how is it different from open for writing?
<mjg> dh`: parallel open of the same file for reading is incredibly common, see -j compiles
<dh`> I mean, sure, it'd be nice to be able to read everything fast and for free
<mjg> for one you don't need to bump any write counts
<mjg> on freebsd for RO open you still pay for refcount grab
<mjg> and read-erlocking around it, which arguably could be eliminated
<mjg> but it definitely does not need to write lock in the common case of a RO oepn
Iris_Persephone has quit [Ping timeout: 265 seconds]
<mjg> note this is not an O_CREAT lookup
<mjg> as in in most cases when you get here the vnode is already fully initialized, along with the backing inode and wahtnot
<mjg> not being able to take advantage of it is a performance bug for years now
<dh`> what doesn't need to write lock? even if you assume the name is in the name cache and the vnode is in the vnode cache, you still need to incref it
<dh`> at an absolute minimum
<mjg> you do with atomics
<mjg> so no need to write lock
<mjg> do it*
<dh`> set the atime
<dh`> update the cache lru
<dh`> etc
<dh`> there's plenty of crap to do
<mjg> i have a sensibly scalable scheme for lru
<dh`> yes, you can cut some of it
<mjg> and in fact even netbsd plays trick on this particular front
<mjg> anyway atime is also a not really a problem
<mjg> note the bsds just set a flag and only deal with actual update on inactive
<mjg> so again by the time you open the flag is probably already set
<mjg> so you got nothing to do
<dh`> it's not that simple
<mjg> it's literally what the code is already doing even on netbsd, exept under a write lock
<mjg> or rather, how it has been done before i got involved in any of this
<mjg> and i'm confident nobody patched it
<dh`> I'm sure nobody did, yeah
<dh`> I tend to be preoccupied with questions about whether it crashes
<mjg> what this should be doing is storing a timestamp on access, only doing the write with atomics if the timestamp is newer
<dh`> the asynchronous timestamp stuff we have is awful
<mjg> as is there is an incredibly retarded problem where the vfs layer grabs a nanosecond-repcision timestamp to update atime possibly 30s later
<mjg> which isl ike LOL
<dh`> I forget where it came from but it's both a maze and has frequently been wrong
<mjg> well you can keep the retarded scheme without the perf impact
<mjg> and you can make a sensible scheme without the perf impact either
<dh`> it needs to be redone sometime
<mjg> but bottom line this can scale sensibly
<mjg> also noatime is quite popular, so there is that
<dh`> it has caused any number of problems, e.g. ls -l in /dev causing all ttys to apparently unidle, which is a gigantic headache on a multiuser machine
<mjg> or perhaps someone(tm) should steal relatime
<dh`> and worse there was a problem at one point where it resulted in inodes not getting written to disk by sync
<dh`> all of which matters a lot more than whether it's a bit slow
<mjg> sure corretness first, but contending like this with 4 workers is not "a bit slow" in 2022
<dh`> perhaps
<dh`> I foresee someone dropping in a bunch of fragile special-case fast path code that only improves anything for artificial benchmarks
<dh`> and makes a mess of everything else and makes it harder to rectify other more significant problems, like opening devices
<mjg> i foresee nobody doing squat in netbsd
<mjg> unless ad@ comes back for somer eason
eck has joined #osdev
[itchyjunk] has quit [Remote host closed the connection]
<dh`> we tend to be more concerned about correctness
Iris_Persephone_ has quit [Read error: Connection reset by peer]
<dh`> e.g. the other day we discovered that O_NONBLOCK silently failed on /dev/ptmx
Iris_Persephone_ has joined #osdev
_xor has joined #osdev
<wxwisiasdf> what
<wxwisiasdf> i have netbsd on my laptop and i've been told that shutting it down while it's doing IO is a bad idea
heat has quit [Ping timeout: 260 seconds]
smeso has quit [Quit: smeso]
gildasio has quit [Remote host closed the connection]
gildasio has joined #osdev
demindiro has quit [Ping timeout: 252 seconds]
gildasio has quit [Remote host closed the connection]
gildasio has joined #osdev
smeso has joined #osdev
vdamewood has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
wxwisiasdf has quit [Quit: Lost terminal]
<klys> okay the w7pro64 vm with my gtx 960 is up and running now on the epyc 7453 box. using debian linux v4.16.18 at the host with kvm.
<klys> not much for storage yet tho, running from 1 480GB sata ssd with w7pro64 on a mechanical disk.
<klys> just may need to clean things up a bit, considering I may have to find some space to though.
freakazoid332 has joined #osdev
zaquest has quit [Remote host closed the connection]
zaquest has joined #osdev
axis9 has quit [Quit: joins libera]
scoobydoo has quit [Ping timeout: 246 seconds]
scoobydoo has joined #osdev
Matt|home has joined #osdev
<klys> and I rebuit linux-4.16.18 with -j30 and it finished before I'd counted to 160.
smach has quit [Remote host closed the connection]
smach has joined #osdev
bgs has quit [Remote host closed the connection]
k8yun_ has quit [Quit: Leaving]
scoobydoo_ has joined #osdev
scoobydoo has quit [Ping timeout: 265 seconds]
scoobydoo_ is now known as scoobydoo
darkstardevx has quit [Ping timeout: 264 seconds]
freakazoid332 has quit [Ping timeout: 260 seconds]
scoobydoo_ has joined #osdev
scoobydoo has quit [Ping timeout: 260 seconds]
scoobydoo_ is now known as scoobydoo
smach has quit []
netbsduser has joined #osdev
heat has joined #osdev
GeDaMo has joined #osdev
m5zs7k has quit [Ping timeout: 265 seconds]
_xor has quit [Quit: brb]
m5zs7k has joined #osdev
CryptoDavid has joined #osdev
tarel2 has joined #osdev
gxt has quit [Remote host closed the connection]
gxt has joined #osdev
tarel2 has left #osdev [#osdev]
Matt|home has quit [Quit: Leaving]
gxt has quit [Remote host closed the connection]
vinleod has joined #osdev
gxt has joined #osdev
vinleod is now known as vdamewood
SGautam has joined #osdev
alphajuliet has joined #osdev
<alphajuliet> so good to be back :)
* alphajuliet rub shoulders with GeDaMo mrvn clever j`ey
* alphajuliet and spank them
<alphajuliet> geist: well i told you you either unban me or im coming
<alphajuliet> cheers
* kazinsal takes a fat bong rip
xenos1984 has quit [Read error: Connection reset by peer]
<Mutabah> alphajuliet: You do know what the general opinion is on ban evasion, right?
<alphajuliet> since when we on #osdev are "general opinion" ?
<kazinsal> tbh I'm technically a ban evader
<alphajuliet> ^
<kazinsal> but I ate a perma from the forums in 2010
<kazinsal> which is three presidential terms ago
<Mutabah> Oof
<alphajuliet> #osdev is like a supreme court ... he doesn't mess with "general opinion"
<Mutabah> alphajuliet: Just behave, alright?
<Mutabah> Also - If you want to be unbanned officially - Provide justification.
<kazinsal> and since then I've gone to university, become a productive member of society, and a local wizard of various subsects of operating systems development
<Mutabah> and check the timezone of the op you PM - Asking at 3AM is counter-productive
<kazinsal> four and eight years respectively
<alphajuliet> Mutabah: i needed to check what's up with the pro's
<Mutabah> kazinsal: Nice
<alphajuliet> also sometimes people say smart things here ... and i learn concepts
<froggey> kazinsal: what did you do to get banned? I'm guessing "be an annoying kid"?
<alphajuliet> which tbh it's almost guranteed that i every do something with
<kazinsal> pretty much, I think I was about 14-15 at the time
<alphajuliet> but that's not the point
<alphajuliet> the point is
<kazinsal> now I'm aged and jaded and a battle-worn artisan of the dark arts of convincing project managers I'm doing more than I actually am
<froggey> mhmm mhmm. peak annoying kid age
<alphajuliet> kazinsal: same
<Mutabah> alphajuliet: Just FYI, there are logs
<alphajuliet> just replace "project managers" with "my mom"
<alphajuliet> well it's not easy to find a job ok ?
<kazinsal> I turn theoretical dollars into practical dollars
<alphajuliet> attaboy
<alphajuliet> kazinsal: if you want to signal that you doing something which you actually don't play this:
<bslsk05> ​play.typeracer.com: TypeRacer - Play Typing Games and Race Friends
<zid> speaking of general opinion, what's the general opinion on feeding trolls? :P
<alphajuliet> but don't be too fast otherwise it will be seems that you are chatting
<alphajuliet> my general suggestion is that you keep it on the 60 70 wpm
<kazinsal> I spend between 2-3 days working from home and I work for a company that isn't insane so I don't have to "prove" productivity by weird software-calibrated metrics
<alphajuliet> sigh im not a troll please stop with this delegitimation tactic i just
<alphajuliet> verbose
<Ermine> You are troll.
<alphajuliet> Ermine: you don't even know me
<alphajuliet> because i don't know you
<Ermine> Knowledge is not symmetrical relation.
JerryXiao has quit [Quit: Bye]
JerryXiao has joined #osdev
<Mutabah> alphajuliet: If you have a question, ask it. If you know the answer to a posted question, provide it. Otherwise, keep quiet.
<Mutabah> Currently - you're just making noise
<kazinsal> technically doesn't really have to be a question - a point of discussion while other people are around and free to Discuss (tm) is a good thing too
<kazinsal> eg. it is too late for me to posit any points of discussion abut I have a few saved up for more populous times
<alphajuliet> Mutabah: what do you think about my chat AI kazinsal ?
<alphajuliet> pretty accurate isn't he
xenos1984 has joined #osdev
<kazinsal> I can confirm my lack of AI-dom
<ddevault> does anyone have some good resources on strategies for managing virtual address space
<bslsk05> ​wiki.osdev.org: Books - OSDev Wiki
CryptoDavid has quit [Quit: Connection closed for inactivity]
alphajuliet has left #osdev [WeeChat 3.6]
Iris_Persephone_ has quit [Read error: Connection reset by peer]
<bslsk05> ​paste.sr.ht: memory.txt — paste.sr.ht
<zid> das very rigid
alphajuliet has joined #osdev
<alphajuliet> why clang says conflicting types ?
<alphajuliet> https://bpa.st/B4ZQ
<bslsk05> ​bpa.st: View paste B4ZQ
<bslsk05> ​paste.sr.ht: memory.txt — paste.sr.ht
<mjg> wtf
<bslsk05> ​lore.kernel.org: [PATCH] fs: remove fget_many and fput_many interface
<mjg> where is the win supposed to be coming from
<mjg> no info what hw is this
<mjg> how was this accepted is beyond me
<mjg> i mean the clean up is probably fine, but the performance claim smells like bs
<alphajuliet> i forgot that getline() is a standard lib
<alphajuliet> function
<bslsk05> ​git.sr.ht: ~sircmpwn/helios: docs/memory.txt - sourcehut git
dennisschagt_ has quit [Quit: https://quassel-irc.org - Chat comfortably. Anywhere.]
dennisschagt has joined #osdev
dennisschagt has quit [Client Quit]
dennisschagt has joined #osdev
isaacwoods has joined #osdev
alphajuliet has quit [Quit: WeeChat 3.6]
SGautam has quit [Quit: Connection closed for inactivity]
carbonfiber has joined #osdev
elastic_dog has quit [Ping timeout: 250 seconds]
elastic_dog has joined #osdev
<mrvn> alpha2023: you should support seekable files for infinite line length without having to allocate huge buffers
Andrew is now known as ay
rurtty has joined #osdev
bazylevnik0 has joined #osdev
frkzoid has joined #osdev
elastic_dog has quit [*.net *.split]
Oshawott has quit [*.net *.split]
Celelibi has quit [*.net *.split]
pieguy128 has quit [*.net *.split]
maksy has quit [*.net *.split]
Irvise_ has quit [*.net *.split]
moberg has quit [*.net *.split]
hmmmm has quit [*.net *.split]
CYKS has quit [*.net *.split]
ptrc has quit [*.net *.split]
MrBonkers has quit [*.net *.split]
rb has quit [*.net *.split]
Ram-Z has quit [*.net *.split]
lanodan has quit [*.net *.split]
alpha2023 has quit [*.net *.split]
outfox has quit [*.net *.split]
Ameisen has quit [*.net *.split]
kkd has quit [*.net *.split]
linkdd has quit [*.net *.split]
jleightcap has quit [*.net *.split]
patwid has quit [*.net *.split]
matthews has quit [*.net *.split]
Affliction has quit [*.net *.split]
xenos1984 has quit [*.net *.split]
heat has quit [*.net *.split]
scoobydoo has quit [*.net *.split]
Raito_Bezarius has quit [*.net *.split]
jjuran has quit [*.net *.split]
k0valski1889 has quit [*.net *.split]
catern has quit [*.net *.split]
Arsen has quit [*.net *.split]
qookie has quit [*.net *.split]
bombuzal has quit [*.net *.split]
remexre has quit [*.net *.split]
DonRichie has quit [*.net *.split]
divine has quit [*.net *.split]
gjnoonan has quit [*.net *.split]
jack_rabbit has quit [*.net *.split]
jimbzy has quit [*.net *.split]
Maja[m] has quit [*.net *.split]
ephemer0l has quit [*.net *.split]
ElementW has quit [*.net *.split]
ckie has quit [*.net *.split]
tomaw has quit [*.net *.split]
basil has quit [*.net *.split]
JerOfPanic has quit [*.net *.split]
bradd has quit [*.net *.split]
JerryXiao has quit [*.net *.split]
corank_ has quit [*.net *.split]
DanDan has quit [*.net *.split]
Gooberpatrol66 has quit [*.net *.split]
pounce has quit [*.net *.split]
janemba has quit [*.net *.split]
renopt has quit [*.net *.split]
seer has quit [*.net *.split]
ids1024 has quit [*.net *.split]
knusbaum has quit [*.net *.split]
ZipCPU has quit [*.net *.split]
koolazer has quit [*.net *.split]
ay has quit [*.net *.split]
rurtty has quit [*.net *.split]
m5zs7k has quit [*.net *.split]
zaquest has quit [*.net *.split]
epony has quit [*.net *.split]
roan has quit [*.net *.split]
bauen1 has quit [*.net *.split]
dormito has quit [*.net *.split]
marshmallow has quit [*.net *.split]
joe9 has quit [*.net *.split]
Goodbye_Vincent has quit [*.net *.split]
Coldberg has quit [*.net *.split]
fkrauthan has quit [*.net *.split]
troseman has quit [*.net *.split]
wgrant has quit [*.net *.split]
night has quit [*.net *.split]
stux has quit [*.net *.split]
justache has quit [*.net *.split]
pie_ has quit [*.net *.split]
vancz has quit [*.net *.split]
mrvn has quit [*.net *.split]
arminweigl has quit [*.net *.split]
Patater has quit [*.net *.split]
Ermine has quit [*.net *.split]
energizer has quit [*.net *.split]
jeaye has quit [*.net *.split]
valeriusN has quit [*.net *.split]
meisaka has quit [*.net *.split]
sjs has quit [*.net *.split]
pg12_ has quit [*.net *.split]
mcfrdy has quit [*.net *.split]
kori has quit [*.net *.split]
thaumavorio has quit [*.net *.split]
genpaku has quit [*.net *.split]
nur has quit [*.net *.split]
dminuoso has quit [*.net *.split]
nisa has quit [*.net *.split]
SanchayanMaity has quit [*.net *.split]
kklimonda has quit [*.net *.split]
Amanieu has quit [*.net *.split]
V has quit [*.net *.split]
Benjojo has quit [*.net *.split]
seds has quit [*.net *.split]
immibis has quit [*.net *.split]
dequbed has quit [*.net *.split]
gaze___ has quit [*.net *.split]
gruetzkopf has quit [*.net *.split]
kazinsal has quit [*.net *.split]
mats1 has quit [*.net *.split]
netbsduser has quit [*.net *.split]
k4m1 has quit [*.net *.split]
\Test_User has quit [*.net *.split]
potash has quit [*.net *.split]
andreas303 has quit [*.net *.split]
JTL has quit [*.net *.split]
klys has quit [*.net *.split]
ozarker_ has quit [*.net *.split]
Emil has quit [*.net *.split]
eschaton has quit [*.net *.split]
moon-child has quit [*.net *.split]
particleflux has quit [*.net *.split]
merry has quit [*.net *.split]
eau has quit [*.net *.split]
j`ey has quit [*.net *.split]
isaacwoods has quit [*.net *.split]
dennisschagt has quit [*.net *.split]
puck has quit [*.net *.split]
chibill has quit [*.net *.split]
Griwes has quit [*.net *.split]
friedy has quit [*.net *.split]
torresjrjr has quit [*.net *.split]
clever has quit [*.net *.split]
amine has quit [*.net *.split]
varad has quit [*.net *.split]
froggey has quit [*.net *.split]
les has quit [*.net *.split]
identitas has quit [*.net *.split]
jstoker has quit [*.net *.split]
Killy has quit [*.net *.split]
ecs has quit [*.net *.split]
relipse has quit [*.net *.split]
AmyMalik has quit [*.net *.split]
mimmy has quit [*.net *.split]
manawyrm has quit [*.net *.split]
corecode has quit [*.net *.split]
vdamewood has quit [*.net *.split]
kaichiuchi has quit [*.net *.split]
pitust has quit [*.net *.split]
hl has quit [*.net *.split]
sbalmos has quit [*.net *.split]
amj has quit [*.net *.split]
klange has quit [*.net *.split]
danlarkin has quit [*.net *.split]
stephe has quit [*.net *.split]
exec64 has quit [*.net *.split]
alethkit has quit [*.net *.split]
milesrout has quit [*.net *.split]
noeontheend has quit [*.net *.split]
ddevault has quit [*.net *.split]
tom5760 has quit [*.net *.split]
cultpony has quit [*.net *.split]
dragestil has quit [*.net *.split]
phr3ak has quit [*.net *.split]
CompanionCube has quit [*.net *.split]
ggherdov has quit [*.net *.split]
XgF has quit [*.net *.split]
mxshift has quit [*.net *.split]
w41 has quit [*.net *.split]
Luci-ghoule has quit [*.net *.split]
Mutabah has quit [*.net *.split]
Stary has quit [*.net *.split]
Rubikoid has quit [*.net *.split]
brenns10 has quit [*.net *.split]
zhiayang has quit [*.net *.split]
snickerbockers has quit [*.net *.split]
ebb has quit [*.net *.split]
kanzure has quit [*.net *.split]
paulbarker has quit [*.net *.split]
LambdaComplex has quit [*.net *.split]
gxt has quit [*.net *.split]
gildasio has quit [*.net *.split]
bazylevnik0 has quit [*.net *.split]
carbonfiber has quit [*.net *.split]
frkzoid has quit [*.net *.split]
ThinkT510 has quit [*.net *.split]
terrorjack has quit [*.net *.split]
aleamb has quit [*.net *.split]
LittleFox has quit [*.net *.split]
Brnocrist has quit [*.net *.split]
dennis95 has quit [*.net *.split]
aejsmith has quit [*.net *.split]
air has quit [*.net *.split]
Bonstra has quit [*.net *.split]
sebonirc has quit [*.net *.split]
alexander has quit [*.net *.split]
woky_ has quit [*.net *.split]
bleb has quit [*.net *.split]
travisg has quit [*.net *.split]
weinholt has quit [*.net *.split]
gdd has quit [*.net *.split]
dh` has quit [*.net *.split]
HeTo has quit [*.net *.split]
ChanServ has quit [*.net *.split]
gdd has joined #osdev
aejsmith has joined #osdev
pitust has joined #osdev
knusbaum has joined #osdev
MrPortmaster has joined #osdev
frkzoid has joined #osdev
rurtty has joined #osdev
carbonfiber has joined #osdev
elastic_dog has joined #osdev
isaacwoods has joined #osdev
dennisschagt has joined #osdev
xenos1984 has joined #osdev
JerryXiao has joined #osdev
gxt has joined #osdev
vdamewood has joined #osdev
m5zs7k has joined #osdev
netbsduser has joined #osdev
scoobydoo has joined #osdev
zaquest has joined #osdev
gildasio has joined #osdev
ThinkT510 has joined #osdev
corank_ has joined #osdev
Raito_Bezarius has joined #osdev
roan has joined #osdev
Celelibi has joined #osdev
marshmallow has joined #osdev
epony has joined #osdev
Oshawott has joined #osdev
k4m1 has joined #osdev
DanDan has joined #osdev
genpaku has joined #osdev
aleamb has joined #osdev
Gooberpatrol66 has joined #osdev
puck has joined #osdev
terrorjack has joined #osdev
ptrc has joined #osdev
bauen1 has joined #osdev
jjuran has joined #osdev
pounce has joined #osdev
k0valski1889 has joined #osdev
jstoker has joined #osdev
identitas has joined #osdev
\Test_User has joined #osdev
Arsen has joined #osdev
qookie has joined #osdev
gjnoonan has joined #osdev
rb has joined #osdev
les has joined #osdev
Ram-Z has joined #osdev
janemba has joined #osdev
dormito has joined #osdev
maksy has joined #osdev
bombuzal has joined #osdev
pieguy128 has joined #osdev
remexre has joined #osdev
nur has joined #osdev
jack_rabbit has joined #osdev
DonRichie has joined #osdev
jimbzy has joined #osdev
seer has joined #osdev
renopt has joined #osdev
joe9 has joined #osdev
divine has joined #osdev
Goodbye_Vincent has joined #osdev
Maja[m] has joined #osdev
Coldberg has joined #osdev
Killy has joined #osdev
Irvise_ has joined #osdev
chibill has joined #osdev
ephemer0l has joined #osdev
hmmmm has joined #osdev
moberg has joined #osdev
fkrauthan has joined #osdev
lanodan has joined #osdev
troseman has joined #osdev
catern has joined #osdev
CYKS has joined #osdev
wgrant has joined #osdev
LittleFox has joined #osdev
outfox has joined #osdev
ElementW has joined #osdev
AmyMalik has joined #osdev
Ameisen has joined #osdev
night has joined #osdev
potash has joined #osdev
Brnocrist has joined #osdev
kkd has joined #osdev
friedy has joined #osdev
ids1024 has joined #osdev
Griwes has joined #osdev
tomaw has joined #osdev
matthews has joined #osdev
patwid has joined #osdev
jleightcap has joined #osdev
linkdd has joined #osdev
alpha2023 has joined #osdev
andreas303 has joined #osdev
basil has joined #osdev
kaichiuchi has joined #osdev
ckie has joined #osdev
dennis95 has joined #osdev
valeriusN has joined #osdev
torresjrjr has joined #osdev
koolazer has joined #osdev
Ermine has joined #osdev
ZipCPU has joined #osdev
air has joined #osdev
exec64 has joined #osdev
alethkit has joined #osdev
milesrout has joined #osdev
ddevault has joined #osdev
tom5760 has joined #osdev
woky_ has joined #osdev
dh` has joined #osdev
thaumavorio has joined #osdev
energizer has joined #osdev
Mutabah has joined #osdev
kori has joined #osdev
mcfrdy has joined #osdev
pg12_ has joined #osdev
meisaka has joined #osdev
sjs has joined #osdev
jeaye has joined #osdev
merry has joined #osdev
gaze___ has joined #osdev
dequbed has joined #osdev
immibis has joined #osdev
Benjojo has joined #osdev
Amanieu has joined #osdev
seds has joined #osdev
V has joined #osdev
SanchayanMaity has joined #osdev
dminuoso has joined #osdev
froggey has joined #osdev
varad has joined #osdev
amine has joined #osdev
corecode has joined #osdev
manawyrm has joined #osdev
mimmy has joined #osdev
stephe has joined #osdev
relipse has joined #osdev
danlarkin has joined #osdev
klange has joined #osdev
amj has joined #osdev
sbalmos has joined #osdev
Luci-ghoule has joined #osdev
mxshift has joined #osdev
XgF has joined #osdev
phr3ak has joined #osdev
cultpony has joined #osdev
w41 has joined #osdev
paulbarker has joined #osdev
kanzure has joined #osdev
ebb has joined #osdev
snickerbockers has joined #osdev
zhiayang has joined #osdev
brenns10 has joined #osdev
Rubikoid has joined #osdev
LambdaComplex has joined #osdev
Stary has joined #osdev
HeTo has joined #osdev
weinholt has joined #osdev
travisg has joined #osdev
bleb has joined #osdev
ChanServ has joined #osdev
Affliction has joined #osdev
nisa has joined #osdev
noeontheend has joined #osdev
Bonstra has joined #osdev
eau has joined #osdev
ay has joined #osdev
j`ey has joined #osdev
JerOfPanic has joined #osdev
gruetzkopf has joined #osdev
kazinsal has joined #osdev
mats1 has joined #osdev
dragestil has joined #osdev
klys has joined #osdev
sebonirc has joined #osdev
mrvn has joined #osdev
CompanionCube has joined #osdev
ggherdov has joined #osdev
moon-child has joined #osdev
Emil has joined #osdev
ozarker_ has joined #osdev
arminweigl has joined #osdev
eschaton has joined #osdev
vancz has joined #osdev
kklimonda has joined #osdev
alexander has joined #osdev
pie_ has joined #osdev
Patater has joined #osdev
stux has joined #osdev
bradd has joined #osdev
clever has joined #osdev
ecs has joined #osdev
justache has joined #osdev
particleflux has joined #osdev
hl has joined #osdev
JTL has joined #osdev
heat has joined #osdev
sm2n has quit [Max SendQ exceeded]
wolfshappen has quit [Max SendQ exceeded]
sm2n has joined #osdev
wolfshappen has joined #osdev
<heat> <heat> mjg, lmao
<heat> <heat> mysterious benchmarks are the most lkml thing ever
<heat> <heat> love the bench
<heat> thanks netsplit
<heat> mjg, have you noticed netbsd rolls some sort of dollar store RCU for the file table?
<mrvn> heat: the array of filedescriptors in the process?
xenos1984 has quit [Ping timeout: 268 seconds]
<heat> yup
<heat> turns out having a lock on that shit is pretty much impossible
xenos1984 has joined #osdev
<mrvn> can't you manipulate that atomically?
<mrvn> What's the data there? Not just a "FileP fds[]"?
<heat> sure
<heat> but you need to grow the table, etc
<heat> you can't just std::vector it
<mrvn> well, you could but that would need std::atomic<std::vector>
<mrvn> You don't need to grow it often so having a dollar store RCU there for growing seems ok
<heat> there's a lot of fuckery involved
<heat> you will need a shit ton of atomic cmpxchg operations I assume
<heat> for instance, fd allocation
<mrvn> But I want open/close to use simple cmpxchg. But when they happen while another thread is growing the table that wouldn't work.
<mrvn> How do you even RCU the array? Make a copy of the whole table on every modification?
<heat> yup
<heat> well
<heat> modifications here = grow or shrink
<mrvn> doesn't work
<heat> then you individually rcu each fd (each index in the table)
<heat> I think that works
<mrvn> One thread starts growing, allocates the memory, starts copying. Meanwhile another thread closes a file but the entry has already been copied. So when the growing finishes the file is back open.
<mrvn> The grower needs to fail and do over.
<heat> right
<heat> i... don't have an answer to that
<mrvn> The file limit is usually something like 1024, or 4k on a 32bit system. Why not just allocate the full size array from the start. can't allocate less than a page anyway.
<heat> it's not a hard limit
<heat> >can't allocate less than a page anyway.
<heat> erm, yes you can
<mrvn> if you SLAB them or something, sure.
nyah has joined #osdev
<heat> of course?
<heat> i'm not getting pages for my allocations lol
<mrvn> I do for anything approaching page size and above. :)
<heat> 8192 is too large of a penalty for each process too
<heat> and it effectively doesn't work
<mrvn> practically there are very few processes with a larger limit. But yeah, 8k per process adds up.
<heat> "practically" in your linux system you mean
<heat> I can very easily raise the limit in pid 1
<heat> in fact, it used to have no limit for me
<mrvn> You can. But why would you?
<heat> 1024 is not enough in a lot of cases
<heat> everything non-trivial
<mrvn> I've had 2 or 3 in my life.
<heat> the cf workers runtime had to raise the limit to like 300k or something
<heat> and it sometimes wasn't enough!
<mjg> heat: the fd hack was copied to freebds
<mrvn> I think a good method might be to mark the functions manipulating the file table and have a method in the kernel to pause all other threads (with waiting for those in the critical functions to finish), then grow the table and unpause.
<heat> NO
<heat> definitely not
<heat> i want to make this as concurrent as possible
<heat> not stop the world
<mrvn> even the growing which you do in exponential steps?
<heat> yes
<heat> even the growing
<heat> in fact, you could just plop a lock around the growing
<heat> that would probably be better
<mrvn> not unless every function takes the lock
<heat> not exactly
<heat> there's some stupid trickery I may need to figure out
<mrvn> yeah, the part where one process closes a file while you grow it
<mrvn> heat: with the RCU you can do all reads without lock while writes will need to use allocate-copy-modify-store.
<heat> i also can't use rcu so :)
<mrvn> So open/cose are slow but read/write keeps on going in parallel
<heat> maybe if I switch my project's name to Linux
<heat> open/close would be slower if they needed to take a lock
<mrvn> why can't you use RCU?
<mrvn> hmm, I think a lock is faster than allocating and copying the table
<heat> no
<heat> that depends on how much contention you have
<mrvn> and how many open files. copying 300k entries will be dead slow
<heat> ctrl+f "Patent"
<mrvn> Do you even use an array? What if I dup2(0, 1000000000)?
<mrvn> 1) not valid where I live, 2) I don't want to know, just in case I ever go to the US. Because knowingly violating is worse than ignorance.
<heat> yes, you use an array
<geist> all of this seems like a real pre-optimization
<heat> anything else is substantially worse
<geist> though if it's for fun, SGTM
<heat> yeah
<heat> i'm yet to try rw spinlocks
<heat> that may work
<mrvn> Isn't the dollar store RCU patent free?
<heat> i guess
<heat> can i penetrate netbsd code and understand what it's doing? that would be a resounding "no"
<mrvn> And doesn't a patent loose it's validity if you don't enforce it? With linux using it that should void any claim.
<mrvn> heat: with "rw spinlock" you mean all operations just take a read lock and do atomic manipulations and growing takes a write lock?
<heat> that could work?
<heat> I was assuming a naiive scheme here
<heat> >With linux using it that should void any claim.
<heat> no, it was given explicitly to Linux for Linux
<mrvn> Sure. You just need to stop any process from messing with the table while you grow it.
<mrvn> Args, that means it's just licensed to them so it's still enforced.
<heat> Linux isn't a blatant patent violation :)
<mrvn> How did Linus accept that? That goes against anything the GPL stands for,
<heat> does it?
<mrvn> the source is GPL but you can't copy it because then you get sue for the patent.
<mrvn> sued
<heat> i think it was licensed to all gpl2+ and lgpl2+ projects
<heat> "In contrast, implementation of RCU is permitted only in software licensed under either GPL or LGPL. Sorry!!!"
<heat> anyway, shame :)
Iris_Persephone_ has joined #osdev
<mrvn> Your opposed to linking some LGPL code into your kernel?
<heat> yes
<heat> anyway, these conversations are stupidly useless
<heat> the outcome is always "yeah. shame" or "fuck patent law"
<bslsk05> ​git.kernel.org: kernel/git/torvalds/linux.git - Linux kernel source tree
<heat> rust on lunix
<heat> \o/
<mrvn> You can always just use state of the art from the 90s where the patents have lapsed by now. :)
vdamewood has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
<heat> i'm on 2001 memory allocation now
<heat> my synchronization primitives are really lagging behind :v
<mrvn> heat: I don't even have threads.
wxwisiasdf has joined #osdev
<wxwisiasdf> good morning operatin the deveopments in a system (probably) o/
<geist> heat: remember focus on features first, optimize later
<geist> and/or design for optimize but you dont have to do it yet
* geist tries to gently nudge heat back from the swirling abyss of endless optimization
<wxwisiasdf> oh non
<geist> nless of course you just want to
<heat> :DD
<heat> in this case new features = optimization as well
<heat> I need to revamp my IO, memory management (as in get a radix tree in for vm objects)
<heat> oh yeah I had a rename(2) thing going on
<zid> have you considered adding CreateWindowEx
<heat> >ui
<heat> lmao no lmao
<Ermine> have you considered adding ebpf?
<heat> yes
<heat> i was thinking about it yesterday
<heat> how cool is it to be able to sample anything? very cool, that's the answer
<mjg> how anything is anything
<heat> they seem to be able to attach ebpf on fucking everything
<heat> including most functions
<mjg> dtrace could already do that 15 years ago
<mjg> no need to get hyped man
<heat> but ebpf is objectively better in 2022
<heat> sorry
<mjg> not saying no
<mjg> just you are getting hyped over a thing you should not
<mjg> what's next, netbsd rocks because it scales better than open
<mats1> major general buzzkill
<mjg> want some ebpf relaities?
<mjg> here is what it does NOT instrument: memset, memcpy etc.
<mjg> because these are coded by hand and don't have a recognized prologue
<mjg> this also means an ez idea like: dtrace -n 'fbt::memset:entry,fbt::memmove:entry,fbt::memcpy:entry /arg2 > 256 && arg2 < 800/ { @[probefunc] = lquantize(arg2, 256, 800, 8); }'
<mjg> will quire you to patch the kernel, which partially defeats the point
<mjg> basically the hype, as you can imagine, dies when confronted with reality
<mjg> heat: uh man + struct slab_cache_percpu_context pcpu[CONFIG_SMP_NR_CPUS];
<heat> LMAO
<mjg> heat: you are rolling wit an array of these, you need this aligned to a multiple of cache line size
<heat> it was my best solution for a really shitty problem
<mjg> you are false sharing memory here
<heat> ok boomer
<heat> that's probably the least of my concerns
<heat> my poll is slow as shit and takes up a good chunk of a make -j4
<heat> did you forget that?
<mjg> 's a one line fix
<mjg> to slab
<mjg> you sound like an old unix geezer right now
<mjg> __atribte__((__aligned(CACHE_LINE_SIZE_MOTHERFUCKER))
<mjg> or steal the handy macro: #define __aligned(x) __attribute__((__aligned__(x)))
<heat> im an old unix geezer
<wxwisiasdf> lol
<heat> i have netbsd installed
<heat> and openbsd for purposely bad code
<heat> and Safety(tm)
<heat> s/purposely/purposefully/
<mjg> here is a tutorial
<mjg> 1. slap somethign together
<mjg> 2. when a mistake is pointed out handwave it away
<heat> when do i start naming things like brelse
<mjg> fair questio
<mjg> n
<mjg> i would say pick a competing project and claim the authors are clueless
<heat> can i write comments?
<heat> i assume that's a no go
<mrvn> mjg: alignas() rocks
<mjg> comments are 100% encouraged, just make sure they don't hep
<mjg> help
<mrvn> and [] attributes instead of __attribute__
<mjg> for example now that your malloc used slab you can semi-accuratey describe the pre-slab state
<mrvn> comment: malloc is bad, causes contention and bad locality.
<mrvn> or is that too helpful?
<mjg> malloc: like calloc, but don't zero out, unless zero out
<mjg> calloc: see malloc
<heat> lmao
<mrvn> mjg: Note: for security reasons zero out anyway, just in case
<mjg> also make something up
<mjg> here is a real historical tidbit
<mjg> a long time the bsds started skipping PIDs < 100 on allocation
<mjg> after a wraparound
<mjg> as in you boot the system, pids roll from 1 up
<heat> yeah
<mjg> but eventually they wrap, and should that happen the alloc routine would do scans only > 100
<heat> that seems wonderfully arbitrary
<mjg> * If the process ID prototype has wrapped around,
<mjg> * restart somewhat above 0, as the low-numbered procs
<mjg> * tend to include daemons that don't exit.
<mjg> i was hanging out in the area(tm)
<heat> what the fuck
<mjg> and knowing the unix shit, i asked is this even true
<mjg> and turns out it is not
<heat> hahahahahahaha
<heat> what the actual fuck
<mjg> booting the system is a fork and exec fest
<mjg> so ye, plenty of holes in the 1-100 range
<mjg> i patch it
<mjg> i take it to an oldu nix geezer, saying the skippin' makes no sense
<mjg> and he literally repeats +/- the above comment
<mjg> convinced the guy to fucking check
<mjg> and he had to concede the comment is not true
<heat> namecheck him
<mjg> now, in principle. it could have been true when it was written, but .... i seriously doubt
<heat> it's also a shit idea
<mjg> it made half-sense since allocation was fucking terrible
<heat> "we could have pids under 100... but meh, don't bother, probs not am i right guys"
sinity has quit [Remote host closed the connection]
<mjg> .. in many cases it had to walk all processes, process groups and sessions to check if given id is free
<heat> ................................................
<mjg> so the person who added the skip had their heart in the right place
<mjg> heat, if you are surprised to find linear scans in an old unix codebase, i suggest you get out while you still can
<heat> im not surprised if openbsd has them
<heat> but i would expect some standards from net or free
<heat> especially free
<mjg> afair they do something less retarded
<heat> come on
<mjg> some form of a hash or somethin'
<mjg> so they have fewer elems to check, but they still have to inspect objects
<mjg> freebsd got a bitmap
<mjg> in fact several
<mjg> one for pid, one for pgrp and so on
<mjg> bit_ffc_at(&proc_id_pidmap, trypid, pid_max, &result);
<mjg> [..]
<mjg> if (bit_test(&proc_id_grpidmap, result) ||
<mjg> bit_test(&proc_id_sessidmap, result) ||
<mjg> goto retry;
<mjg> trypid = result + 1;
<mjg> bit_test(&proc_id_reapmap, result)) {
<mjg> }
<mjg> not claiming perfection, but good enough to not be a problem right now
<mjg> i think what this needs to do is 1. increase pid range 2. distribute it across domains
<mjg> netbsd has some crazy code here, but it does avoid the lineare scans
<mjg> with obj inspection
<heat> fuck
<heat> i think linux is a lot smarter than this
<mjg> is it?
<mjg> i think they have sparse arrays(?)
<mjg> which is still in the same ballpark
<mjg> but no wi'm curious
<mjg> now i'm
zaquest has quit [Ping timeout: 252 seconds]
xenos1984 has quit [Ping timeout: 246 seconds]
<mjg> ye no man
<mjg> the code is pretty crazy
<heat> ahhhhhhhh
<heat> it uses a radix tree
<heat> this is smart
<mjg> for pids?
<heat> yup
<bslsk05> ​elixir.bootlin.com: idr.c - lib/idr.c - Linux source code (v6.0) - Bootlin
<mjg> i'm saying why do you think this is smart for pids
<mjg> should you go up to pid_alloc you will find they still take a global lock to handle it
<heat> idea being that you tag entries and propagate this tag up the tree
<mjg> but what you really want is distributed state
<heat> so you can intelligently find out what pids are used
<heat> man
<heat> stfu
<heat> this is great
<mjg> where you can get away with disjoint locks
<heat> better than a linear search
<mjg> you do realize the bitmap approach is almost not inspecting shit to find the free ids, right?
<mjg> as in you find a free id in the first word you inspect most of the time
<mjg> also i can make it scale better than the above by making than one copy
<heat> until you wrap around
<mjg> sure, there will be corner-case calls which have to walk more
<mjg> if this ever turns out tobe a problem, a 2 level bitmap can be used
<mjg> so more get skipped at the same time
<heat> wrapping around is not "just a corner case"
<mjg> i'm not saying it is
<mrvn> The bitmap becomes horrible when it becomes full. With 16bit pids that is a real danger.
<mjg> i'm saying the cases where you have to inspect a lot are rare
<heat> as in, I bet most long running freebsd systems out there have wrapped around
<mjg> you wrapped
<mrvn> And with 32bit pids a bitmap is a big chunk of memory.
<geist> i guess posix says a pid must be int?
<geist> and thus probably 32bit?
<mjg> i don't know what limit in imposes
<mrvn> geist: pid_t
<geist> you can make it 64bit then
<mjg> heat: i'm saying how many consecutive ids do you expect taken to be skipped over
<heat> geist, no, posix doesn't specify a limit iirc
<geist> theh you can just roll a counter
<heat> sortie has 64-bit pids
<clever> geist: one thing that bothers me, is that the old 16bit limit got turned off by default, and pid's are no longer something i can just quickly type
<mjg> tooling is not ready for a 64 bit pid
<clever> /proc/sys/kernel/pid_max
<mrvn> geist: you mean drop the bitmap as 64bit will never roll over?
<mjg> waitpid does not handle it
<clever> but i can clamp it with this tunable
<geist> mrvn: yeah
<Ermine> That's a lot of processes
<mrvn> clever: on the other hand 16bit pids basically roll over during boot.
<mjg> but neither the tooling nor typical intefaces can handle such big ids
<geist> yah
<geist> we have problems in fuchsia with our 64bit kernel object ids getting smashed in posixy apis
<clever> mrvn: another thought i have, is how zfs manages its inode table, each node in the indirection tree, has a count of free slots for all leaves under that node
<geist> or at least existing code that assumes 32bit pids
<mjg> 's what i said
<heat> yes
gog has joined #osdev
<clever> you could use a similar set of stats for an index into a pid->pointer table
<heat> sortie has a bunch of patches for third party sw on this
<heat> he basically converts to intmax_t and prints it
<mrvn> clever: basic improvement when you need to search for gaps in a tree.
<mjg> you can do that in your custom os if you want, you can't if you have to run unix apps
<heat> you can
<heat> patch your packages?
<heat> send 'em upstream
<mjg> every single one
<heat> it is how it is
<mjg> someone sometime will have to fix all of it
<clever> mrvn: the trick with zfs, is that it repurposed a field that is meant to count how many leaf nodes exist, to instead count holes within the leaf
<geist> fight the man!
<geist> dont let the monoculture of linux win!
<mjg> bue in the meantime 32-bit space is mor than plenty enough, so i see 0 incentive by people to do the work
<heat> you realize freebsd is going to need to upstream a bunch of fixes for all the ports right now, right?
<heat> clang straight up broke autoconf
<heat> it will be in the next release
<mjg> how do you justify 64 bit pids to joe port author
<heat> it's portable
<mjg> *today*
<heat> well, sorry
<mrvn> mjg: we don't want to reuse pids because that might confuse users and make logs harder to correlate
<heat> no need to justify 64-bit pids
<heat> say "convert pids to intmax because it's the only portable way to print a pid"
<heat> easy
<geist> well this gets exactly to the point of all of this hobby os stuff: why bother implementing anything but a linux clone nowadays?
<geist> answer is because you fucking wanna
<mjg> that's not what i said
<heat> I FUCKING WANNA IMPLEMENT A LINUX CLONE
<geist> sure but functionally its the same thing. any time yo udo something that's not like linux there's patches to be done.
<mjg> in fact quite the contrary
<mjg> there is the question however for far did they go in linux in the pid space
<mjg> they seem to be 7 digit now?
<mjg> at least
<geist> i think it's a runtime tunable?
<mjg> kernel.pid_max = 4194304
<mrvn> heat: you could use a PRIpid / SCNpid macro like PRIdLEAST32
<geist> yah
<mjg> wut, they actually went full int here
<geist> i remember with some of the fuchsia builds we were actually hitting the limit at build time if it was set to something low like 16k
<mjg> hmmm
<mjg> ok then, that's somehin'
<mrvn> geist: odd number. I've seen signed short plenty of times but 16k ...
<geist> i dont remember if it was 16k. it was something fairly low but reasonable
<mjg> i wonder if they tried booting with starting point set arbitrarily to 4 mln
<mrvn> Can you set kernel.pid_max lower than an existing pid?
<mjg> i'm not tryingthat
<mjg> :>
<geist> i assume it resizes some table somwehere, i guess
<mrvn> same here. But should be safe if the limit is only used when allocating new pids.
<mjg> have fun tsting
xenos1984 has joined #osdev
<mrvn> geist: they have a radix tree someone said earlier. So no table with 4 billion entries.
<geist> hmm, where are you reading it?
<geist> ah but it might affect the size of the radix
<geist> maybe it reindexes it based on that
<mrvn> could be. Already said I'm not trying it.
<heat> radix trees auto-resize
<geist> ah /proc/sys/kernel/pid_max
<mrvn> geist: kernel.pid_max is the sysctl syntax for that path
<geist> ah seem smost of my linux boxes are 4G, except a 32bit linux x86 box where it's 32768
<mrvn> Hard to have 4 billion threads with 4GB of ram max.
<geist> raspberry pi 4 is set to 32k as well
<gog> hi
<heat> hi
<mjg> is that the same kernel?
<mjg> roughly anyway
<geist> no, so it may be an older vs newer kernel
<geist> or it could be some distro tunable
<bslsk05> ​stackoverflow.com: What are the factor on which PID_MAX depends in Linux - Stack Overflow
<heat> also distro tunable yeah
<heat> you can write that
<mrvn> Linux ryzen 5.7.0-0.bpo.2-amd64 #1 SMP Debian 5.7.10-1~bpo10+1 (2020-07-30) x86_64 GNU/Linux
<mrvn> $ cat /proc/sys/kernel/pid_max
<mrvn> 32768
<gog> 4194304
<heat> 💯
<mrvn> €
<gog> Linux icemoon 5.19.7-1-MANJARO #1 SMP PREEMPT_DYNAMIC Mon Sep 5 10:16:57 UTC 2022 x86_64 GNU/Linux
<geist> ah okay that makes sens e(in the stackoverflow)
<heat> gog, really?
<heat> manjaro?
<geist> the default value being set to something based on how it was compiled (max cpus, etc)
<geist> no distro shaming!
<gog> yes really
<gog> linux is linux
<mjg> the answer is from 2016
<heat> manjaro is the most shamable distro ever
<heat> i would rather run openbsd than manjaro
<mjg> i have a 4 thread laptop with UINT_MAX limit here
<gog> what's wrong with manjaro
<heat> the devs are actively incompetent
<geist> sure but it's based on how it was compiled, now how many cpus are present
<gog> ah yeah
<gog> nah they're not actively incompetent
<bslsk05> ​manjarno.snorlax.sh: Manjarno
<gog> they're passively incompetnet
<heat> hey maybe I should run netbsd on my rpi zero 2w
<heat> for the funzies
<gog> idk i don't have problems with it, it works fine, i just don't update
<gog> never update
<heat> i'm unfortunately running manjaro there because it was the only distro with proper support for it at the time
<geist> as a side note i think that CONFIG_SMALL style switch is something i'm going to need to eventually put in LK. i've been dreading doing it, but there are so many places where having some sort of overall vague notion of 'this is an embedded system' vs 'this is bigger, has paging, etc' would be really helpful
<mjg> +1
<gog> ¯\_(ツ)_/¯
<gog> i'm incompetent too
* geist pets gog
<gog> i shouldn't be a programmer
* gog prr
<mjg> have you seen programmers
<wxwisiasdf> programmers
<geist> competance is not a requirement for programmers
<heat> unix
<mjg> it is discouraged
<mjg> pid_max = min(pid_max_max, max_t(int, pid_max,
<mjg> PIDS_PER_CPU_DEFAULT * num_possible_cpus()));
<mjg> 6.0
<wxwisiasdf> progammers 1 nanosecond after downloading 1.3TB of node_modules
<gog> lmao
<gog> we need to step it up at work, we've only got 800MB of node modules in our apps
<mjg> haha node.js goes brrr
<wxwisiasdf> i've seen node_modules go around the gigabytes on some webserver thingies
<gog> my payment processing refactor went into staging for tests today
<gog> it crahsed and burned
<gog> i gotta fix some things
<mjg> you got staging environment?
<gog> yes
<mjg> normally this means it is not representative of production
jjuran has quit [Ping timeout: 268 seconds]
<gog> it's not
<mjg> different versions and configs
<mjg> right
<mjg> welcome to programming
<wxwisiasdf> lol
<mjg> you get to fix bugs twice
<gog> it works... when the card is valid and 3DS verification succeeds
<heat> u guys support nintendo switch
<gog> yes
<heat> good
<mjg> nintendo snitch
<heat> u know its a microkernel right
<heat> cry bsd man
<heat> yor tiny mind does not compreehend the macroest microkernel
<mjg> you do realize ns uses some freebsd code
<mjg> afair parts of ther network tsack
<heat> lmao
<heat> that's so vanilla
<gog> i'm gonna write the microest macrokernel
<mjg> think about that next time you plan your pay to wn game on it
<heat> "so i stole bsd's network stack" is like 90% of operating systems
<gog> i know that apple already did
<mjg> heat: now that is a meme and it is most likely not true
<heat> even fucking windows stole it
<wxwisiasdf> heat: "so i stole bsd" - every os ever
jjuran has joined #osdev
<mjg> windows had userspace tooling partially taken from bsd
<gog> ms eventually wrote their own
<mjg> but i have no idea about the kernel side of things
<heat> they stole kernel too
<gog> they stole about half of NT from IBM
<wxwisiasdf> they stole IBM from IBM
<heat> bsd steals bsd all the time
<wxwisiasdf> wait but who stole who
<gog> that's why NTFS and HPFS have the same ID in MBR
<heat> what
<gog> yeh
<gog> look it up nerd
<wxwisiasdf> no way
<mjg> using nerd as a derogatory word on this channel is 90% funny
<sbalmos> the other half they stole from DEC
<mjg> and 3rd half?
<sbalmos> they wrote from scratch
<gog> ballmer wrote it in parts while on various benders
<sbalmos> DEVLEOPERS DEVELOPERS DEMENTERS!
<wxwisiasdf> isn't windows basically a "what if we took everything and put it with a nice UI"
<gog> no because it doesn't have a nice UI
<wxwisiasdf> i mean they stealin' from KDE so uh oh
<sbalmos> Win2k/2k3 was a nice UI
<wxwisiasdf> and OS/2 UI
<heat> what
<heat> kde stole everything from windows
<heat> kde is windows
<wxwisiasdf> windows stole from kde
<heat> no
<gog> it's a dialectic of theft
<heat> >isn't windows basically a "what if we took everything and put it with a nice UI"
<heat> no
<heat> the kernel is really nice
<gog> yes
<wxwisiasdf> okay fair enough
lg has quit [Ping timeout: 265 seconds]
<sbalmos> DPL/IOPL stuff is interesting
<wxwisiasdf> virtualizaion and directx12 nice
<gog> i always point out that under all the shit the NT kernel is a piece of true software engineering
<heat> it is
<heat> large codebase full of history
<wxwisiasdf> adds in file manager tho
<heat> does it have cruft? absolutely
<heat> but so is life
<gog> it's been around for 30 years
<sbalmos> I wish I could've skipped straight from Win2k3 straight to Win11
<heat> freebsd is cruft galore
<gog> of course it's gonna be crufty and rotty
<gog> the codebase i work on is 6 years old and it's got bad cruft everywhere
<sbalmos> and I still miss Windows Classic mode
<sbalmos> anything >1 production release old has cruft
<wxwisiasdf> what about having cruft from day 1
<sbalmos> that's why you're supposed to delete the git branch and start from scratch once you really know WTF you're writing
<wxwisiasdf> lmao
carbonfiber has quit [Quit: Connection closed for inactivity]
dude12312414 has joined #osdev
Iris_Persephone_ has quit [Read error: Connection reset by peer]
Iris_Persephone has joined #osdev
lg has joined #osdev
GeDaMo has quit [Quit: Physics -> Chemistry -> Biology -> Intelligence -> ???]
<gog> yes
<gog> rewrite from a fresh repo
<gog> cast your fuckups into the gaping maw of history
<heat> love me some gaping
<gog> lmao
<gog> stop
<heat> geist, re: memory allocation, did you read my comments regarding linux's buddy allocator yesterday or so?
<geist> a little bit
<heat> I think it can work quite well if you go down the magazine route like they did freakin 20 years ago
<geist> i mean buddy alloctor, yes
<heat> basically to avoid excessive splits and merges of higher order buddies they just maintain an order-0 percpu cache and allocate/free from there
<geist> makese sense
<heat> like bonwick's idea for slabs, but for page allocation
<geist> right
<heat> which I was going to mention as an interesting idea if you want to go down the physmap route before we diverged into kasan
<heat> otoh, i dont know how many of these concepts actively make a difference in a microkernel
<geist> nope it's 100% on the radar of things to do
<geist> of which there are essentially an infinite list of, of which like 5 people are working on
<heat> what is?
<geist> replacing the PMM with something else, probably a buddy allocator
<heat> ah
<geist> since everything depends on everything else, it gets difficult to replace things in motion
<geist> which is of course the hard part about hacking on an OS that's actually Shipping
<geist> different universe re: hobby oses and open source stuff
<heat> wdym replace things in motion?
<geist> but yes, it's on the radar of things to do
<heat> it's not like replacing the backend for alloc_page() can actively break any consumer
<heat> especially on a ukernel
<geist> alas, yes it can
<heat> why? perf regressions?
<geist> notably any regressions of performance, space, etc would have to be Justified
<geist> and there's eom bootloader stuff in play, how many arenas does the kernel have? how would that intersect with the pmm
<geist> not unsolvable, but as is usual it's more complicated
<heat> eom?
<heat> you mean oem?
<geist> sentence fragment?
<geist> oh oh i see. 'some'
<heat> ah
<geist> when i mean in motion: imagine you're a car driving down the freeway at full speed, and need to replace subsystems of it without it stopping
<geist> it's harder that way, alas
jjuran has quit [Ping timeout: 250 seconds]
jjuran has joined #osdev
<sbalmos> that's why there are emergency pull-over lanes and flashing warning lights in cars. ;)
<gog> good luck changing a tire
<zid> That's the easy bit I've seen it
<zid> you find a ramp to go up onto two wheels with
<zid> and do it from the window
<jimbzy> gog, That's why I always make sure my auto insurance has roadside assistance.
_xor has joined #osdev
<mjg> part of the problem iwth serious perf work
<mjg> there was (is?) a yolo-ed page replacement policy in linux
<mjg> G wrote a better version and had to provide extensive data to show it is better
<mjg> while the previous yolo could just go in, cause why not bro
<zid> see, piece of piss
<gog> jimbzy: we need computer program insurance
<gog> computer programs can fail catastrophically and do untold damage to life and capital
<jimbzy> What would the invisible hand think about such a thing?
<jimbzy> We have to consult the Elder Scrolls of Capitalism!
<mjg> there was a cancer treatment device which was killing people
<gog> yeh because it was miscalibrated
<gog> and overdosing people
<bslsk05> ​'"NTFS really isn't that bad" - Robert Collins (LCA 2020)' by linux.conf.au (00:48:04)
<gog> therac 25
<mjg> ye i was blanking on the name
<mjg> got theranos :D
<gog> lmao
<mjg> anyway, re the above talk, lolo syscalls which are the bane of existence
<gog> i'm gonna watch that
<gog> after watching this other thing
<gog> and getting hummus
frkzoid has quit [Ping timeout: 260 seconds]
freakazoid332 has joined #osdev
SGautam has joined #osdev
<geist> i always thought NTFS itself is a pretty solid design
<geist> or at least was very solid at the time it was written (1988)
<geist> and that's because it's functionally a new implementation of FILES-11 from VMS
<mjg> i have no idea
<geist> from skimming through it i do like his general respect and niceness
<geist> he makes a prety good point at the end about not just coming in with guns blazing about performance issues
<clever> mjg: the therac 25 also had software bugs that where a non-issue on a previous model, due to hw interlocks
<clever> mjg: the new model had no hw interlocks
<mjg> geist: what i don't get from the talk is why there are no flamegraphs or an equivalent
<mjg> is there no visibility of that sort for end users?
<geist> of what? the kernel?
<mjg> yea
<geist> oh i dunno
<mjg> he shows a strace-like output and speculates what's up
<geist> i think there is a lot of ability for itnernal tracing but maybe without debugging info it's not that useful
<geist> but i honestly have no idea
<mjg> ultimately i liked not rolling with "windows == bad bro"
<mjg> but i hoped this would be a kernel dev which could share what is actually going on there
<mjg> speakin' of unix, just got fsck segfualt
<mjg> :>
theruran has quit [Quit: Connection closed for inactivity]
carbonfiber has joined #osdev
bauen1 has quit [Quit: leaving]
netbsduser has quit [Remote host closed the connection]
<heat> mjg, probably because flamegraphs aren't very useful without symbols or source code
<gog> i hold all computer software with about the same degree of contempt
<heat> mjg, calling it "lolo syscalls" isn't particularly correct
<gog> as opposed to what other kind of software
<heat> as I said a few days ago, "windows is not unix"
<heat> it just isn't
<heat> Close() flushing file buffers is something you would never do in unix cuz muh performance
<heat> zfs devs would have a stroke on this level of data integrity and sync
<heat> :v
<heat> nt isn't bad, it's just that unix people don't know how to use nt
<mjg> the guy expicitly gives an example of opening and closing the same file back to back to do different hings on it
<mjg> that's lolo syscall usage
<mjg> and flamegraphs are perfectly useful without source code
<heat> you're talking about that open() + close + stat + etc?
<heat> where they had stupid syscall sequences?
<mjg> yes
<heat> well, that's a thing they can get away with in unix
<heat> but not windows
<heat> because windows is windows and not unix
<mjg> what could have happened is open ..... do shit .... close
<mjg> i'm sayi8ng the fact it was not like that resulted in lolo syscalls
<heat> yes
<mjg> in particular stat-equivalent several itmes
<heat> i thought you were talking about the syscalls themselves
dude12312414 has quit [Ping timeout: 258 seconds]
<heat> how would you use a flamegraph without source?
<heat> "ah, yes, 0xffffffff802a2324 takes up 5% of samples"
<mjg> you are conflating lack of source code with inability to resolve symbols
<heat> well, source and symbols
<heat> they're not giving you their symbols
<mjg> without symbols that's unusable
<mjg> see my previous question about end-user profilability (if that's a word)
<heat> hrm, they do have to give out some symbols for debugging right?
<heat> I don't know what subset of symbols that is
<heat> probably the kernel API
<heat> >there was (is?) a yolo-ed page replacement policy in linux
<heat> 1) yes, it was easy to add it there because there was no page replacement before
<heat> 2) is, MGLRU is only coming out in the next release
<heat> maybe it's in -next already, idk
<mjg> the point of the remark was that doing things right(tm) suddenly requires way more work if there is a total crap in place already
<heat> yea
<heat> particularly socially, even if you don't have much technical debt
<heat> how to convince people that you're right and they're wrong and bad and poopy
<j`ey> heat: it is in -next
<heat> poggers as the kids would say it idk im not a kid
<j`ey> you kinda are
<heat> stfu
<zid> heat is a sperm
<zid> (compared to geist)
<heat> shit u calling him old now?
bauen1 has joined #osdev
<heat> j`ey, have you rewritten realms in rust yet
<heat> actually rewrite arch/arm64
<heat> under the guise of "llvm supports arm64 so why not"
<j`ey> yep, thats what the r in realms stands for
<j`ey> rust ealms
<heat> what does the r in rust stand for?
<j`ey> realms
<heat> woah
<j`ey> gnu is not a unikernel
<zid> suppose heat is true
<heat> i am truth
<gog> i am lies
* kof123 .oO( zelda 2 error has cousins, ok... )
<kof123> small world
dude12312414 has joined #osdev
nick64 has joined #osdev
<nick64> When a VM causes an SMI, does the SMM know that the CPU is virtualised and the SMI came from a code executed in the VM?
<heat> causes as in inside the vm?
<nick64> Yeah
<heat> if you get an SMI in a hypervisor you'll go to the hypervisor's SMI handler
<heat> not the host
<nick64> What does "SMI in a hypervisor" mean here?
<gog> the guest shouldn't even be able to trigger an SMI anyway afaik
<nick64> Why not?
<heat> wrong
<gog> oh?
<heat> if you outb 0xb1 (intel's register that you poke and generate an SMI) under a hypervisor, that SMI is handled entirely inside the hypervisor
<nick64> I know little to none about SMI, but I was reading SMI can come from 3 class of reasons, 1. the hardware related interrupts, 2. Traps from the kernel, 3. SMI by the SMM itself due to pre-configured timer
<nick64> I was referring to case 2
<heat> it has nothing to do with the actual SMM and SMIs your hardware (as in, non-hypervisor'd) has
<heat> yes outb 0xb1 is an example of a way to trap into an SMI
<heat> usually, that is
<heat> this is all "here be dragons", there's no CPU defined mechanism to SMI IIRC
<nick64> heat: When you said "handled entirely inside the hypervisor", do you mean all SMIs raised by the VM cause a VMExit by the VMX hardware components of the physical CPU?
<heat> unless there's some way in the local apic? possibly
<heat> no
<heat> they do not cause a vmexit
<heat> maybe I should've used under here
<nick64> If it does not cause a VMExit, how could the hypervisor possible handle the SMI?
<nick64> possibly*
<heat> SMIs generated under a hypervisor by a hypervisor are handled under a hypervisor
<heat> SMI handlers are regular code under the hypervisor, in SMRAM (system management ram)
<nick64> No I mean SMI traps generated in the Guest VM kernel
<nick64> What happens there?
<heat> I've just told you
<heat> whether vmx has any extension to do the SMI jump without the hypervisor, I don't know
<heat> (as in, without vmexiting to the hypervisor)
<nick64> "SMIs generated under a hypervisor by a hypervisor are handled under a hypervisor"
<nick64> Okay, what about "SMIs generated under a hypervisor by the code running in the non-root VMX partition"?
<heat> but in theory it doesn't need to
<heat> that's exactly what i've told you
<heat> maybe im not being clear here
<heat> SMIs generated by code under a hypervisor run under the hypervisor
<heat> is that clearer?
<nick64> I didn't understand "code under a hypervisor run under the hypervisor"
<heat> dude
<heat> if you set up smm under the hypervisor using a normal method
<heat> as in, setting up smram, etc
<heat> and then trigger an SMI
<nick64> Not sure what you meant by "under the hypervisor"
<heat> that SMI will be handled in the SMRAM
<nick64> You mean before the control is passed to the VM?
<heat> "non-root VMX partition"
<nick64> Oh
<heat> whether there's a vmexit, maybe? I don't know if vmx knows where smram is
<heat> but the SMI is handled in the "vm"'s SMRAM itself
<heat> and you're under an emulated SMM
<nick64> Okay, so let us say if I run a CentOS machine on intel hardware, with Virtualbox installed in it, and run Ubuntu VM inside it, and there is a kernel module in the Ubuntu VM that does something that creates an SMI, the handler for that will be in the hypervisor kernel and not of the VM?
<nick64> "and you're under an emulated SMM" This!
<nick64> That clarifies
<heat> now, if you get an actual SMI from your non-hypervisor thing, you will vmexit straight to SMM
<nick64> So hypervisors have emulated device code for SMM?
<heat> SMM are handled by firmware
<heat> SMI*
<heat> it's hard to link you code because this is stupidly complex but
<nick64> So you are saying it will not be handled by the Ubuntu VM's bios.efi that I configured in the virtualbox config of Centos, but the actual Bios in the flash of my gaming rig?
<heat> it will be handled by your bios.efi if it was i.e an outb 0xb1
<heat> if you for some other reason get a trap into REAL SMM, it will be your actual firmware that handles it
<nick64> Is there any scenario where I can get a trap to real SMM from the Ubuntu VM?
<heat> yes
<heat> an easy one is that some other CPU trapped into SMM
<heat> when one traps, the others follow suit
<j`ey> huh?
<nick64> But I cannot control "some other CPU" from the vCPU of Ubuntu VM
<heat> or $chipset-specific-smi-reason
<heat> j`ey, yeah, SMIs are system-global
<heat> either everyone is in SMM or no one is
<j`ey> interesting
<nick64> I am wondering if I can deterministically make a trap to the real SMM from Ubuntu VM
<heat> i dont know
<heat> i believe not
<heat> but again, it depends on your chipset and whatever
<nick64> Also my main question that was supposed to follow is, does SMM know where the SMI came from? (VMX non-root vs VMX-root)
<heat> if you get an SMI if your temp reaches 90C, you can overheat your cpu inside your vm up to 90c
<nick64> haha
<nick64> But I don't get to control the data there
<nick64> But I got the point
<heat> >does SMM know where the SMI came from?
<heat> what SMM are we talking about here?
<heat> the real one or the one under the hypervisor
<nick64> From some trap caused by the VMX non-root kernel (aka under the hypervisor in your terminology)
<nick64> That doesn't matter, is the ACTUAL SMM virtualisation aware is my query
<heat> no
<heat> i assume you either get some vmexit or get sneaked back into the VM
<heat> let me check
<nick64> I see. If that is the case, there has to be some code in the virtualbox source code devices directory that handles either the emulation or the sneak back, which I am curious where it'd be
<nick64> Can you give me an example of a software triggered SMI apart from the explicit `int 0xblah`
<nick64> err. `out 0xblah`
<heat> no
<heat> you usually don't trigger SMIs, SMIs trigger you
<heat> joke intended, but also true
<nick64> SMI triggers SMM, but Kernel can trigger an SMI right?
<nick64> Any one of the following can make the CPU go to SMM right? 1. SMM timer event, 2. SMI from some component in chipset, 3. Some trap from Kernel to SMM
<heat> it can but you usually dont
<heat> I don't know what an SMM timer event is
<heat> 2) is always the right choice
<heat> there's no "trap to SMM" instruction
<heat> anyway the answer for the "do you go back to vmx operation" is yes
<nick64> "SMM runs in the form of interrupt handlers that are triggered by timers or access to certain memory, registers, or hardware resources."
<heat> 31.14.2 of the volume 3 intel sdm
<heat> that's because you can trigger SMIs from the local APIC
<heat> which im not getting into
<heat> but
<heat> you don't do it, it's stupid and useless
<nick64> Which is why I am curious
<nick64> From a security point of view
<heat> from a security POV this is all bad and horrible
<nick64> :)
<heat> SMM is not secure and I don't know what the person writing that blog post was smoking
<heat> microsoft is literally pushing for a solution that doesn't involve SMM and SMIs
<nick64> Intrinsic security vs baked insecurity
<heat> in fact, they landed it
<nick64> VBS?
<heat> visual basic scripts is not the answer
<heat> PRM
<nick64> Isn't VBS the answer to security? :P
<heat> SMM will really only be needed in the future for authenticated variables
<heat> hopefully, that is
rorx has quit [Ping timeout: 244 seconds]
<nick64> I remember this privilege separation being mentioned here before
<heat> what privilege separation?
<nick64> PRM is privilege separation of the SMM monolith right?
<heat> sure, i guess
<heat> it's supposed to slim down the SMM shitshow into a much smaller shitshow
<heat> SMM is one of the worst ideas of x86
<nick64> Would be interesting target to scrutinise
<heat> and it has fucking competition
<nick64> Against LPE from Windows local process to PRM
\Test_User has quit [Quit: .]
<heat> some platforms just give you SMM for free if you reach ring0
<heat> others make you sweat a little
wxwisiasdf has quit [Ping timeout: 246 seconds]
<nick64> Competition? Isn't the software side of SMM upto the vendor/osdev to write/rewrite? And doesn't this PRM thing not depend on the SMI capabilities provided by intel?
<heat> no and no
<heat> no operating system dev writes SMM code
<nick64> Oh! I assumed PRM would be some glorified SMI handler
<heat> it's None Of Your Business
<heat> >PRM thing not depend on the SMI capabilities
<heat> No.
<heat> that would beat the purpose
<clever> nick64: my understanding is that smm is part of what your bios provides, and does things like ps2<->usb emulation, by trapping ps2 access
<nick64> I mean, I thought PRM would be something with footprint on the SMM as well as the NT Kernel, with the bulk of code concentrated towards nt kernel
\Test_User has joined #osdev
<heat> clever, used to. recent platforms don't do that shit
<heat> the PRM is just a way to call external, firmware code
<heat> that you used to do by outb 0xb1
<heat> but now you just call it directly, or in ACPI, and it runs in ring 0
<nick64> So PRM is not glorified SMI handler, PRM is glorified kernel interrupt handler that makes a lot of SMI workflows obsolete?
<heat> no
<heat> there are no interrupts at play
<nick64> How does PRM kick in?
<heat> call
<heat> you should read the pdf
<heat> if you're so curious
* nick64 saving to Notion
<nick64> Looks like this PDF was already there in my todo reads :D
SGautam has quit [Quit: Connection closed for inactivity]
<heat> mjg, are you up for a vfs correctness challenge?
<mjg> heat: sure
<heat> make this stable in netbsd
<mjg> wut? :D
<heat> sorry
<heat> freebsd I mean
<mjg> it is stable on zfs and tmpfs
<heat> i did the brainfart
<mjg> i have not checked ufs
<mjg> i/m not touching hte latter
<mjg> did you actually run into something?
<heat> no
<heat> I was talking with dh` yesterday about this and he told me it's super possible your shit is still broken
<mjg> no
<mjg> i'm aware of the prog for some time now, i even posted how illumos has the rpoblem
<mjg> 3 years now?
<mjg> heat: i think yo uwant to check rename in netbsd :->
<heat> i have
<mjg> ufs?
<heat> no
<mjg> have you seen the comments?
<heat> i was looking at the vfs
<mjg> right
<mjg> i support adding the rename lock, but ENOTIME
<mjg> as for realities of what's going on where
<mjg> ufs_gro_rename
<mjg> * ufs_gro_rename: Actually perform the rename operation.
<bslsk05> ​bxr.su: Super User's BSD Cross Reference: /NetBSD/sys/ufs/ufs/ufs_rename.c
<mjg> 372 goto whymustithurtsomuch;
<heat> love the gotos
<mjg> 492 /* XXX WTF? Why purge here? Why not purge others? */
<mjg> 493 cache_purge(fdvp);
<heat> you know, is this really needed?
<mjg> 526#if 0 /* XXX */
<mjg> 527 goto whymustithurtsomuch;
<mjg> 528#endif
<mjg> 529 goto arghmybrainhurts;
<heat> why does their rename take 300 parameters
<mjg> someone(tm) had a meltdown
<heat> i would assume rename for a simple fs would just be unlink() + link()
<mjg> so to be clear, nobody familiar with the rename fuckery and right mind will ever claim their rename is correct, if they have one
<mjg> you can state though if there are stress tests which fuck it up and i'm unaware of any for freebsd
<mjg> the prog you linked was already used to
<mjg> ... find bugs in illumos, dragonflybsd and netbsd
<mjg> and afair it crashes openbsd
<mjg> freebsd survives no problem
<mjg> given the above commentary, you think netbsd's rename is bug-free? ;->
<heat> apparently
<mjg> 546arghmybrainhurts:
<mjg> 547 UFS_WAPBL_END(mp);
<mjg> 548
<mjg> 549ihateyou:
<mjg> 550 return error;
<mjg> in all seriousness, what on earth is going on with these gotos
<mjg> you would not see my writing anything of the sort in actual code
<mjg> 850 /* XXX Is this correct? */
<mjg> 851 return (VTOI(vp)->i_size == 0);
<mjg> sounds like the code is well commented :->
<heat> where is it
<mjg> just scroll though the file i linked above
<heat> <mjg> comments are 100% encouraged, just make sure they don't hep
<heat> <mjg> help
<heat> that's what I was looking for
<mjg> :)
<mjg> right
<mjg> 268 * A virgin directory (no blushing please).
<heat> why do half their comments start with XXX
<heat> omfg
<mjg> someone should report to the HR epartment
<mjg> maybe a fan of porn given the above
SpikeHeron has quit [Quit: WeeChat 3.6]
<mjg> that file is a work of art
<mjg> 698 return EIO; /* XXX Panic? What? */
<mjg> what?
isaacwoods has quit [Quit: WeeChat 3.6]
<heat> lmfao
<gog> what?
<mjg> what?
<gog> XXX comment indicates an extremely egregious hack
<gog> in my experience
dude12312414 has quit [Remote host closed the connection]
<mjg> normally XXX means that sepcial attention is needed, sometimes due to a hack
<gog> XXX gog
<mjg> heat: that said, how is rename on onyx
<mjg> heat: does dirconc fuck you up? are you at the level of illumos?
<heat> it does
<heat> im debugging it
gildasio has quit [Remote host closed the connection]
<mjg> i take it you don't have the rename lock yet
<heat> I do
<mjg> heh
<heat> now im debugging a deadlock
<mjg> do you take it?
<mjg> :)
<mjg> ok
gildasio has joined #osdev
<heat> previously I had a problem where the old dentry got unlinked and then the rename got confused as parent = null
<heat> but now I return ENOENT to that
<heat> which I guess works?
<mjg> copy paste some comments man
<mjg> and goto labels
<mjg> goto fuckinghell;
<mjg> stop_moving_shit_you_twat:
<heat> goto fuck_off_you_stupid_orphan
<gog> goto bed
<mjg> deadlock_here:
<heat> you know your renames look a lot worse
<heat> linux's look clean-ish
<mjg> that's plausibly true
<mjg> bsd vfs api is fucking stupid
<mjg> which already adds bullshit to sepcial case for
Iris_Persephone_ has joined #osdev
<mjg> but i don't know linux to be particularly clean anywhere
<heat> do you like SAVESTART?
<heat> :P
<mjg> i have a wip patch to remove it
<mjg> the problem is there are 2 lolo filesystems which do something with it
<mjg> so will have to look at them first
<mjg> which is a significant deterrent
<mjg> they are likely already wrong, so it will be several hours of bugfixing before i get to the actual thing
<mjg> that's ext2fs and some other crapper
<mjg> i have to note though, rename locking can be done in a manner which does not serialize it per mount point
<mjg> despite a rename lock
Iris_Persephone has quit [Ping timeout: 264 seconds]
<heat> ext2fs is great
epony has quit [Ping timeout: 252 seconds]
<heat> although it has some sequences of code that look odly familiar to linux
<mjg> the one in freebsd? i'm not familiar with either
<heat> yes
<mjg> i had a look at that rename in there and found it uses vfs_relookup
<heat> i used it as a reference
<mjg> which is where the woes come from :[
<heat> you guys changed the names of the functions?