<mrvn>
mjg: I imagine the macro is empty when you disable preemption
<mrvn>
mjg: in your code: "((bufctl *) ret)->flags = 0;" Why are you have an array of the wrong type?
<mrvn>
s/are/do/
<mrvn>
looks like a bad attempty to c++-ify old C code too.
Arthuria has quit [Remote host closed the connection]
<dzwdz>
> KVM internal error. Suberror: 1
<dzwdz>
should i be worried
<dzwdz>
i'm not sure if this is a bug i should report or if i should just ignore it
nyah has quit [Quit: leaving]
<mrvn>
if you can reproduce it then report it
<heat>
its your problem
<heat>
you're doing something stupid like trying to execute on non-existent memory or MMIO or some similar shit
<heat>
and yes, the error message isn't the best. don't blame me
levitating has quit [Ping timeout: 248 seconds]
<mrvn>
it really isn't.
levitating has joined #osdev
<heat>
similar errors can also happen when you're using fancy instructions on MMIO (that KVM doesn't know how to emulate), or possibly touching non-existent memory too (can't remember)
mctpyt has quit [Read error: Connection reset by peer]
<bslsk05>
โwww.youtube.com: Touhou Music: IRON ATTACK! - Cavalrymaid - YouTube
<zid`>
Amorphia: People say it's scary to go into the woods with me, but they never think how I feel, I have to walk home alone.
<Amorphia>
zid`: lmfao
<bnchs>
mew
<lav>
mew
<lav>
the transfem urge to make an esolang
nyah has joined #osdev
<FireFly>
yes, this checks out
<bnchs>
:3
<FireFly>
considering a number of people I knew in the freenode #esoteric back in the day :p
<moon-child>
oh no
<moon-child>
am I trans now
<zid`>
I wrote a befunge interp once
<zid`>
do I need to buy stripey socks
<lav>
yes.
<Amorphia>
lav: glad you like Bernays, it does indeed "go hard"
sortie has quit [Quit: Leaving]
sortie has joined #osdev
<zid`>
oops just tasered myself playing with my lighter absentmindedly
<zid`>
gog give me sympathy
<zid`>
GeDaMo won't do it
<zid`>
lol fuck, I have a burn hole in my finger
<gog>
aw
<zid`>
If my finger drops off I am blaming GeDaMo so it's okay
<mjg>
luller
<mjg>
truller
mahk has quit [Ping timeout: 268 seconds]
mctpyt has joined #osdev
mctpyt has quit [Ping timeout: 246 seconds]
bradd has quit [Ping timeout: 255 seconds]
<bnchs>
i hate vague OS documentation
<mjg>
:p
<zid`>
stop using vagueOS then
<mjg>
ObliqueOS is the new hotness mate
<bnchs>
trying to figure out how to list a directory in OS-9, the documentation says there's a directory bit for opening the directory file, but doesn't tell me how to use that to list the contents of a directory
<bnchs>
apparently you open the file with the directory bit, and then READ from it to get a 32-byte record per every read that represents a file inside that directory
<bnchs>
thank you vague documentation
<bnchs>
anyway mew
<gog>
mew
<zid`>
maybe we should all just play dark souls instead
<bnchs>
maybe we should all just relax and cuddle
<zid`>
in like a sex way?
<bnchs>
no
<lav>
mew
<bnchs>
in a wholesome way
<zid`>
sex is wholesome if you do it a certain way
<zid`>
not like I do it in, in the woods
<kof123>
"These games are linked through their setting and an overarching cyclic narrative centering around fire"
<kof123>
so what you are saying is there is an os bonfire, where if you fail once, you have to try to retrieve that code's soul
<bnchs>
what is worse than making an OS?
<klys>
paging
<klys>
in general
<kof123>
watching someone else make one?
<bnchs>
reverse engineering an existing OS, and trying to make an OS based off of that, trying to replicate everything as accurately as possible
<klys>
corollary: building over a stereotype of how os's work in textbooks and promulgating a result with zero optimization, ie. building a worse OS.
<kof123>
does that mean you expect to run "original" binaries?
<gog>
i love building a worse os
<bnchs>
kof123: yes, binary-compatible
<bnchs>
it's less like an OS in my case, but sorta like WINE, but shit and for an obscure OS
<klys>
gog, have you been working on an accounting package?
<gog>
me?
<gog>
no
<klys>
out of curiosity
<gog>
my job is web development
ilovethinking has joined #osdev
<ilovethinking>
hello
<klys>
hi
<ilovethinking>
what could be the reason for pages being skipped when virtual mapping?
dutch has quit [Quit: WeeChat 3.7.1]
<klys>
are you getting skipped pte's or pde's?
<ilovethinking>
pte's
<Mutabah>
Defined "skipped"?
<klys>
do they point to physical ram below the detected total?
<ilovethinking>
Mutabah: idk my kernel is trying to access something that is non present in bss and i'm sure i map bss
<ilovethinking>
or my linker script is fucked
<klys>
well rule that out too, print your pointers
dutch has joined #osdev
<klys>
bss mapping is generally done via the sbrk() system call on paging systems
<klys>
which means you'll be calling your paging algo for a userland page fault
<ilovethinking>
wait i should pagealign the virtual address right
<sham1>
If you want to have a pointer to the page, yeah
<ilovethinking>
okay cool
<ilovethinking>
anyways i just noticed random hugepages appear
<ilovethinking>
and it's
<ilovethinking>
weird
<ilovethinking>
i don't map anything as hue
<ilovethinking>
huge*
<ilovethinking>
could there be something premapped by limine?\
<klys>
so print the associated pte if you can find it, presumably there is a way to debug those
<ilovethinking>
i'll just see where's it's mapped and when
biblio has quit [Quit: Leaving]
<ilovethinking>
oh wait i forgot i can;t do shit because my stack isn't mapped
<ilovethinking>
so no outb
<ilovethinking>
kekw
<lav>
debugger
* gog
attaches to lav
<lav>
you want to run me until i hit the breakpoint?
<gog>
no i'm gonna deatch and quit early
<gog>
XD
<zid`>
I'm picturing gog like one of those little fish that follow sharks
<zid`>
remora
<zid`>
That's what I'm calling my debugger
<gog>
neat
<gog>
blรฅhaj os
<sham1>
And when you hit a breakpoint, you print "the program is now sleeping with fishes"
<ilovethinking>
hmmmmm turns out whenever i try to jmp to my actual kernel start from asm it pf's
<ilovethinking>
interesting
brynet has quit [Ping timeout: 246 seconds]
<mrvn>
ilovethinking: So same problem as days ago.
<mrvn>
fix your page table
<mrvn>
just go back and read the log for all the help you will get
<ilovethinking>
mrvn: no, it literally happens at the start of my program
<ilovethinking>
way before i put entries in the pt
<bslsk05>
โwww.youtube.com: Russian Doomer/War Song Collection 1 - YouTube
bgs has joined #osdev
Matt|home has quit [Ping timeout: 260 seconds]
xenos1984 has quit [Quit: Leaving.]
<mjg>
i randomly remembered a geezer anecdote, i figure i will share
<mjg>
pretty funny
<mjg>
there was a 4 numa domain machine running a $workload
<mjg>
geezer flipped memory policy from first touch to round robin
<mjg>
0 checks how it affects the workload
<mjg>
but must be good cause "it will balance memory allocation across domains"
<mjg>
you wont believe it, but it destroyed performance. previously almost all memory traffic was landing in the local domain
<zid`>
weird that
<mjg>
now about 3/4 has to come across interconnects. both ways. for all 4 domains
<mjg>
ikr
<zid`>
you'd think allocating them worst first would be best
<mjg>
i should proposed that
<mjg>
like literally every time the farthest away
<mjg>
anyhow
<mjg>
i can't stress enough how much geezer reasoning is typically a joke
<zid`>
I assume the generic strategy is best to worse, unless you know in advance something will be rarely accessed in which case you push it to keep close less occupied
<zid`>
or do it as an eviction
<zid`>
last-accessed goes out, fresh thing takes its spot
<mjg>
there is no real good strategy if the workload does not do proper affinity on its own
<mjg>
however, least bad in practice tends to be first touch and fallback to round robin in case of shortage
<zid`>
yea I was talking about the on its own part
<heat>
ngl round robin sounds awful
<mjg>
it is
<mjg>
but then again: geezer
<zid`>
maybe it's to prevent erm
danilogondolfo has quit [Quit: Leaving]
<zid`>
like if you used it as a stack
<heat>
if you're going for something better than first touch you probably want to set up NUMA and core afinities
<zid`>
that your early entries just go into the fast shit, then stay there forever untouched, and your 'work' is all on the 'top' of the stack, which is in the slow shit
<zid`>
so random better in that case
<mjg>
speaking of affinity, rust FEARLESS CONCURRENCY of course fails again
<heat>
i would guess that NUMA in a single process is a bad idea
<mjg>
there is no support for any of it inside
<mjg>
heat: it is questionable but can work
<mjg>
anyhow thea ffinity shit is why people tend to try to avoid multisocket boxen
<zid`>
that's an obvious solution
<zid`>
"just have so much ram you don't need to cross domains"
<zid`>
bit of a cop-out though eh
<mjg>
and so much core count
<heat>
actual questions: 1) what happens to the page cache? 2) what happens to anything that touches shared state (i.e core stuff like malloc, libc, whatever)
<mjg>
in the *process*, as far as i know nobody has any provisions for internals to be numa-aware
<zid`>
I assume, a fuck load of QPI nonsense
<mjg>
if anything there is code which ends up shafting you if you get there
<mjg>
you are basically on your own trying to work around libc
<heat>
provisions where?
<zid`>
yea you'd be using malloc + numa aware flags
<heat>
freebsd kernel? libc?
<zid`>
err mmap*
<zid`>
not malloc
<mjg>
heat: any bsd or linux, in userspace
<mjg>
it's all yolo all the way down
<heat>
yay yolo
<heat>
zid`, i guess some of that could work if you supply a numa aware malloc
<zid`>
malloc1, malloc2, malloc3 etc :D
<heat>
slab and slub work pretty well with numa AFAIK
<heat>
mostly because they gate slabs to nodes
<heat>
sharing memory will ofc still universally suck because NUMA
<mjg>
dude even freebsd has mostly operational numa in kenrel allocator
* kof123
deep professional voice "but what does solaris do?"
<mjg>
do you know who does not? SOLARIS :S
<kof123>
that's a good question random observer
<zid`>
slowaris is bad!?
<zid`>
I don't believe you bro
<zid`>
why do you always have to lie like this
<mjg>
which is pretty funny given that part of their schtick was hardware which you put together like lego
<mjg>
they do ok affinity for userspace, but kernel internals are all bullshit, mostly domain 0
<heat>
you know who also does not? Onyx
<heat>
because who the fuck uses NUMA
<zid`>
everybody who buys amd?
<heat>
boohoo
<heat>
deport all AMD users to north korea
<heat>
fuck em all, Intel only baby
<zid`>
but not newer intels, those are also superglued togther
<zid`>
sapphire rapids or whatever
<mjg>
brah
<heat>
oh yes that's too new, too new
<mjg>
SPARC FOR LIFE
<mjg>
fuck you and your itanium
<mjg>
who even uses that
<heat>
any sort of asymmetry really, fuck that shit
<heat>
should be ILLEGAL
<mjg>
i got a great name for big little
<zid`>
heat can we add delay cycles to my L3
<mjg>
sortix.ONYX
<mjg>
hehe!
<zid`>
so there's no asymmetry across cores
<zid`>
just round it all up
<heat>
why is onyx little
<mjg>
du -sh says it is
<heat>
i'm fairly sure the kernel is a fair share larger
<mjg>
is it?
<mjg>
ok you got me, i geezered this one and did not actually measure before talking shit
<zid`>
mjg where did you even get this 'geezer' slang you're using
<zid`>
I think you're using it to mean cowboy?
<mjg>
> The meaning of GEEZER is an odd, eccentric, or unreasonable person (usually a man);
<mjg>
especially : an old man.
<mjg>
that would be the meaning
<kof123>
he's not anti-wizard, he's anti fake-wizard
<mjg>
i need a word for a harmful twat, typically older
<zid`>
boomer
<mjg>
if not g then what
<mjg>
ok boomer
<zid`>
manager
<kof123>
ceo
<heat>
mjg, I have around 170kloc vs sortix's 60kloc
<heat>
my kernel is a lot larger lol
<zid`>
How many once we expand the templates
<zid`>
30-40 million is my guess
<heat>
you want binary size?
<heat>
my kernel barely uses templates
<zid`>
yea that's why I lowballed at 30M
<mjg>
does not sound legit
<heat>
what doesn't?
<mjg>
are you comparing against that dev sortix version
<mjg>
with the network stack and hookers?
<heat>
yes
<mjg>
aight
<mjg>
then i stand surprised
<mjg>
zid`: hey zid, where did i get "aight" from
<zid`>
it's a contraction of 'alright'
<mjg>
i tis, but where from
<zid`>
which is a contraction of 'all right'
<heat>
mjg is an african american man
<mjg>
THE WIRE MOTHER^W
<zid`>
sheeeet
<zid`>
I saw clay davis in something on TV the other day
<mjg>
:]
<heat>
mjg, do you remember when I said sortix didn't have a page cache and shit? it doesn't have a page cache and more
<zid`>
Every time I see an actor from the wire I always say "Oh, that's that guy from the wire"
<mjg>
heat: i don't :)
<mjg>
heat: that sounds horrible
<heat>
well, it doesn't have a page cache
<mjg>
aight, noted
<zid`>
he plays Bob in Cocaine Bear
<mjg>
as great as the wire is, there are very few quotable moments
<heat>
as I've repeatedly told you, i'm the only person in #osdev who obcesses over their toy OS's performance and "doing things properly"
<mjg>
compare to the sopranos
<mjg>
heat: does seem so
<heat>
people are mostly happy that things "work"
<mjg>
heat: don't act like i don't need to repat the same shit to you several times
<mjg>
repeat
<heat>
are you joking
<mjg>
ok, quick quiz
<zid`>
7
<mjg>
what OS has DOORS
<zid`>
quick answer
<heat>
your largest nitpicks on my shit are "make the fast path and slow path very different and separate, and make the slow path into a separate noinline function"
<heat>
mjg, freebsd
<mjg>
you clearly forgot about your lack of cacheline alignment for per-cpu structs
<mjg>
which i'm confident i mentioned at least twice
<mjg>
and other stuff
<zid`>
mjg: Everything is cache line aligned in my kernel.
<heat>
mjg, i'm fairly sure i've done that
<zid`>
Because my allocator can only deal in pages.
<mjg>
#define CACHE_SIZE 20
<zid`>
SUPER aligned
* kof123
lights incense next to doug16k idol "he didn't mean it, he knows not what he says"
<bslsk05>
โgithub.com: Onyx/slab.h at master ยท heatd/Onyx ยท GitHub
<mjg>
heat: aight, it is there, but i had to tell you at least twice.
<heat>
you told me once and I told it wasn't a bottleneck, you told me again and I fixed it
<heat>
and now I'm telling you it still isn't a bottleneck
<mjg>
brah
<mjg>
you are poulling off a geezer play here
<heat>
8 CPUs will not die over a little cacheline sharing
<heat>
this is not a geezer play
<heat>
i'm not running on 80 core xeons man
<zid`>
big iron optimizations
<mjg>
basic code higiene for core primitives includes making sure they don't have trivially avoidable problems
<mjg>
to that end, see remarks like the above
<heat>
yes but 1) my time is limited
<zid`>
every time mjg moans about your alignment you should worsen it by the next prime
<mjg>
i do concede at 8 cpus you will have tons of other problems and the above will probably not come up
<heat>
2) my time is limited and I have way more serious problems
<zid`>
char pad[47];
<mjg>
watch less tik tok
<mjg>
zid`: ikr
<heat>
i don't watch tiktok, I watch youtube shorts
<mjg>
i'm gonna create GeezerBSD
<zid`>
that's called freebsd
<heat>
OpenBSD
<mjg>
openbsd is worse than that
<heat>
actually
<heat>
geezerbsd would be what, Net?
<zid`>
freebsd is only for old men
<zid`>
so freebsd is geezerbsd
<heat>
i had the impression net was geezer-er than free
<zid`>
nobody uses the other bsds for anything, that makes them for hippies, alternative lifestyle people
<mjg>
i'm not gonna shit on netbsd today
<mjg>
i will note there was a particularly bad openbsd talk once
<mjg>
someone decided to take a look at firefox performance
<heat>
freebsd is for weirdos, netbsd is for oldies, openbsd is for <redacted>
<mjg>
and the talk bragged how they managed to get some speed ups without any profiling tooling
<heat>
and the furtive dragonfly, so easily forgotten
<mjg>
as mentioned numerous times, you can get poor man's profiling in few h
<zid`>
can I install dragonfly on my quantum fireball
<mjg>
which would be also sufficient to profile hte above
<mjg>
zid`: is it on a blockchain
<zid`>
what
<mjg>
the answer is Logan Paul
<heat>
brian rose for london mayor
<zid`>
you guys are high or something idk
<mjg>
PRIME
<mjg>
zid`: how old are you
<mjg>
you act like a person born after > 1990
<mjg>
heat: so you prefer logan or jake paul
zid` has left #osdev [#osdev]
<mjg>
i don't know if we have a beef here
<heat>
hmmm
<heat>
that's a hardie
<mjg>
you already lost
<mjg>
there is one legit answer and it is obvious
<heat>
jake is probably the least-worst
<mjg>
tbh i have no idea about the j twat
<mjg>
i only heard about the l twat after the japanese video scandal
<mjg>
and recently after the cryptozoo scams
<mjg>
which were glorious
<heat>
jake has a pseudo boxing career
<mjg>
l does not?
<mjg>
that entire youtuber boxing thing is perplexing
<heat>
idk, does he?
<heat>
oh shit yeah ofc
<heat>
he fought floyd
<mjg>
i like this civilized topic
<heat>
it is civilized, no need to shit on openbsd now, or old people that don't understand NUMA
<heat>
or my fast paths
<mjg>
brah
Arthuria has joined #osdev
<mjg>
teel you what
<mjg>
i'll go entire march without shitting on solaris
<heat>
i'll tell you what hank
<mjg>
if someone baits me i'm gonna ignore
<mjg>
liek the system never released
<heat>
SunOS wasn't that bad
<mjg>
i got few days to shit on it
<mjg>
but wont comment
<heat>
actually, Sun was a great company
<heat>
and this is objectively true
<mjg>
so logan is a handshome guy, is not he?
<heat>
no
<mjg>
do you have his merch
<mjg>
or rather, how much of it do you hav
<mjg>
e
<heat>
logan is too old for me sorry
<mjg>
so you are into guys
<geist>
shitting on other things is boring
<mjg>
it's ok, it's 2023
<geist>
that can be dome anywhere, lets talk about new things please
<mjg>
fuchsia is crap!
<geist>
again, shitting on things is boring
<mjg>
sorry, misread
<mjg>
i thought you meant shitting on old stuff :)
<mjg>
so, here is something positive
<mjg>
i was looking at implementing malloc in userspace
<heat>
ohno
<mjg>
normally they all roll with per-thread caching
<mjg>
but perhaps this can be done per-cpu without atomics
<heat>
yes you can do percpu now
<mjg>
rseq
<heat>
with linux rseq
<heat>
yeah
<mjg>
right
<heat>
i believe tcmalloc has a mode for that too
<mjg>
so i'm thinking defo worth looking into
<geist>
hmm, yeah you're right. hmm
<mjg>
should be a great win in terms of memory usgae
<heat>
last I looked at tcmalloc on a Real Workload(tm) it had some major live/deadlock issues due to bad spinlocks
<geist>
indeed. probablu worth looking into how the rseq stuff works
<heat>
because abseil spinlock is pretty much just a mutex with some spinning on it
<heat>
i have no idea if it behaves well for Google because of their kernel patches or if the Real Workload(tm) was doing something wrong
<heat>
i've heard there's a switch_to syscall internally
<mjg>
any user actual spinlock, as in no fallbak to futex or equivalent, is prone to this problem
<mjg>
unless the workload is tuned to avoid it
<mjg>
known since 90s
<mjg>
and fixed too
<heat>
yes, but it's kind of like an adaptive mutex thing
<geist>
yah the usual reason why if someone says they want to create their own spinlock in user space, they probably dont
<mjg>
right
<heat>
the problem is that you can't really spin on something in userspace ATM
<heat>
i've theorized some ideas for a way for the kernel to pass thread info back down to userspace
<heat>
like a simple struct thread_info {uint scheduled;};
<mjg>
a production-quality idea would be great here
<heat>
the real problem would be that you would need to copy_to_user on every thread switch
<mjg>
right
<geist>
unless it's some sort of shared page
<mjg>
i had a funnty hack idea: reserve a register
<mjg>
for this metadata
<heat>
hmm yes, I've also thought about a shared page
<mjg>
shared page is the standard idea
<mjg>
the problem is what do you do when thread count gets out of hand
<mjg>
there is literally hundereds of thousands
<heat>
right
<geist>
not sure htere's a mechanism for this on linux style systems, but for zircon style (ie, anonymous VMOs with a handle to it) its fairly easy to share data between kernel and user via a vmo
<mjg>
there is not per se
<geist>
ie, 'hey kernel i made a 4k vmo for you to put stuff in'
<heat>
sure, just mmap an fd
<mjg>
but you could argue the vdso thing does it
<mjg>
to export timehands
<heat>
that thread count problem is interesting but I'm not sure if its actually a problem
<geist>
well, mmap a fd would be a real PITA for something super fast, wher eyo uhave to pin it forever, etc
<mjg>
look for a cheap cop out, which may be a dead end in the long run, you can just reserve some VA
<geist>
yah
<mjg>
and map pages as new threads pop up
<mjg>
and wire it in kernel in a way which can't be modified by user
<mjg>
as in no magic munmap or mmap over it
<heat>
like, you could argue that reserving a 4K page for every thread is like getting around the mlock rlimit
<mjg>
et voila
<mjg>
4k for every therad?
<heat>
yep
<mjg>
i thought 1 bit for every thread
<geist>
i've generally thought having an implicit page per thread mapped somewhere would be generally useful for all sorts of things
<mjg>
it is either on or off
<heat>
that sounds like a bad interface
<geist>
even just putting small data in it for syscalls so the kernel can avoid user_copy
<heat>
i.e you want to expose something else
<mjg>
i never thought this out, just spitballing here
<geist>
then obviously the issue is dealing with multiple things trying to use the same page, and how it gets subdivided, etc
<heat>
and fwiw I don't think 4K for every user thread is horrific bad
<froggey>
hmm. why does rseq_cs have both cpu_id and cpu_id_start?
<geist>
same. i mean you already burn at least 4K per thread for the first page of a stack
<geist>
now in a 16K page system the overhead changes, but then that's also a known 16K page thing
<heat>
the problem is that ideally you would count it for mlock limits, and that's not backwards compatible
<heat>
the real problem here is that *somehow* you need to keep the memory alive
<mjg>
so how will you know as the locking primitive where to find whether the particular lock owner is running
<mjg>
you said one page per thread, how do youget to it
<mjg>
from an arbitrary thread
<heat>
easy, mutex->pthread_owner->thread_info
<mjg>
and how do you know that wont fualt
<mjg>
sounds like uucopy needed :X
<heat>
you don't, but you also don't really care
<mjg>
?
<heat>
and it there's no reason why that would happen give that page is not swappable
<heat>
given*
<mjg>
i'm saying in the time window between you finding the lock is held
<mjg>
and derefing the addr
<mjg>
the target thread could have exited
<heat>
right
<mjg>
and the page could have been whacked
<heat>
that's part of the problem
<heat>
you would need some persistance
<mjg>
so uucopy?
<heat>
persistence?
<heat>
idk what uucopy is
<heat>
uucopy - no-fault memory-to-memory copy
<heat>
no
<mjg>
the problem with the "persistence" thing is that it twill mess with real world progs which do a lot of thread creation/teardown
<mjg>
unless you do it rcu-style
<mjg>
things are getting messy real quick here
<heat>
yes, it's not an easy problem to solve
<geist>
yah uucopy is one of those things i want to do in zircon soon too
<mjg>
so that uucopy thing would do the trick
<geist>
it's an excellent thing for the vdso to do
<mjg>
provided it comes from vdso
<mjg>
right
<heat>
why?
<mjg>
which part
<heat>
why would it do the trick?
<heat>
it would not do the trick, and it would be slow
<mjg>
it would guarantee you either get the data you wanted
<geist>
why would it be slow?
<mjg>
or EFAULT
<heat>
e.g consider page reuse
<heat>
slow: you take a trap
<geist>
(just to make sure we're on the same page)
<mjg>
well i agree you could read bullshit
<geist>
you only take the fault if it's a bad page
<mjg>
but this is where things like seq counters come in
<mjg>
heat: normally it would be the same cost as a call to memcpy
<mjg>
basicaly perfectly tolerable
<heat>
I've thought about just letting the libc deal with it and then let it do some rcu-ish-thing, but a problem is that this page must not swap EVER
<geist>
right think of it as a memcpy with an implicit fault handler registered by the kernel (ie, a user_copy in the kernel)
<geist>
which is why it'smore or less free if you implement it in the vdso
<heat>
ideally a linux patch would be to get_user_pages a struct page and pin that stuff
<heat>
but again, then questions arise regarding mlock rlimit
<mjg>
this one is not that bad
Turn_Left has joined #osdev
<mjg>
if you roll with my idea, of say 1 bit per thread, you can have a dedicated rlimit how many threads you can spawn based on it
<mjg>
you still wire the page, but it is not general purpose and does not count towrds mlock
<mjg>
and it wont go unchecked
<mjg>
2 pages gives you 8192 threads, which should be enough for almost everybody(tm)
<mjg>
and not be a big deal
<mjg>
not claiming perfection on this one
<heat>
yea, but then imagine you want to add more stuff to it
<heat>
more bitmaps?
<mjg>
sure extensibility is a problem, but see the above memory usage concern
<mjg>
it is a tradeoff
Left_Turn has quit [Ping timeout: 252 seconds]
<heat>
and how would you map this thing? I don't think automagically mapping a bitmap would fly
<mjg>
i would totes auto do it, just like vdso
<mjg>
maybe have it off until first exra thread spawns
<geist>
yah in the zircon model you could hand user space a handle and then itโs user spaces problem to map it
<mjg>
implementation detail
<geist>
you could i guess basically mint a fd to something that you mmap
<mjg>
key initital thing to fiture out is how this would look like in an alredy running process with thousands of threads
<mjg>
my proposal adds little memory usage, but adds some ping pong to all the on/off transitions
<mjg>
in fact you may want to sack an entire byte instead of a bit to avodi atomics
* mjg
makes a sad face
<Ellenor>
wow, this is running quickly
<heat>
lets say we did a struct thread_info {u8 oncpu;}; and packed it tightly in a page by the libc
<heat>
i think this would work
<mjg>
this is basically what i said
<mjg>
except it needs to be able t roll with more pages
<heat>
yes, but by the libc. libc would basically specify the format to the kernel
<mjg>
uuh
<geist>
how does user space thenknow what cpu its on?
<heat>
and dynamically allocate
<geist>
doesnโt it need to know that instantaneously to utilize this?
<mjg>
geist: for adaptive spinning you don't need to know which cpu
<mjg>
geist: you only know whether it is on or not, so a bit is enough to convey this
<mjg>
need to know*
<heat>
yes, if another thread is running you already know it's not on yours
<geist>
kk, was going to get to the fact that you canโt easily know the cpu youโre on for all arches
<geist>
and of course the instant you know yitโs out of date, etc
<CompanionCube>
mjg: inb4 loophole: shit on illumos instead
<mjg>
in that spirit the on/off bit is also out of date.. so
<geist>
kk, makessense
<mjg>
CompanionCube: nah mate, i don't play dirty
<geist>
CompanionCube: no lets not. i planon trying to clamp down a bit on long โshit on thisโ sessions here. i dont think we generally need that around here
<mjg>
ye as much as i enjoy it, it is not produtive to the channel, if you will
<mjg>
unless someone explains how to do better instead, which is rarely included
<geist>
๐
<mjg>
there should be #osdev-rant :)
<heat>
geist using an emoji moment
<heat>
๐ฅ
<geist>
๐ค
<mjg>
no poop emoji from me, even though you probably expect it
<lav>
๐ฑ
<heat>
colon three but zoomer
<geist>
mjg: if youโd like. there have been spin offs of this channel quite a few tims, including an osdev-offtopic
<geist>
none of them survived the transition to libera
<heat>
sure they did
<mjg>
geist: i can't be arsed to reate a channel, and already happen to have a vent space elsewhere :)
<heat>
osdev-offtopic is offtopia pretty much
<mjg>
that said, if smoeone else makeso ne, i'll join
<mjg>
i really don't need to shit on solaris *here*
<heat>
hi mjg, it's called "my PMs"
<mjg>
:)
<geist>
i do like that about discord, but i think most discord servers over fork the channels. so you end up with a bunch of highlyspecific channels that no one wants to use
<geist>
oryou always get punted onto another one
<heat>
anyway all these ideas warrant some kind of fun patching. it's just that it's kind of hard to divise an upstreamable solution
<mjg>
well let's say the cpu state bit is not feasible for the time being
<heat>
you would need the libcs and the kernel to all agree on a sane solution
<heat>
haha
<mjg>
even then some degree of speculative spinning will probably help
<mjg>
heat: haha... ha?
<mjg>
by some degree i mean not much
<heat>
spinning on what?
<mjg>
you see thel ock is taken, cpu_pause()
<mjg>
do that a few times
<mjg>
if still taken, go to the kernel
<mjg>
determining sensible 'few' is the problem here
<mjg>
this can very easily backfire
<heat>
yes
<heat>
abseil and glibc pthread_mutex does it based on time AFAIK
<mjg>
now that i wrote it, i'm confident the real long time solution will include a way to convey thread state to the kernel at the time of a trap
<mjg>
but right now you can't do it through user memory due to the prohibitive smap cost for this purpose
<heat>
right
<mjg>
hm
<heat>
something something shared page ;)
<mjg>
hmmm
<mjg>
ye i'm leaning towards an idea that userspae would get a page marked as "special shit goes on here mate"
<mjg>
and it will copy various actual funcs into it
<mjg>
rseq-style, except more general
<mjg>
[i have barely skimmed rseq so maybe i mised something]
<heat>
copying functions doesn't sound good to me
<mjg>
key is that you could easily check that RIP falls within the special area
<mjg>
and only then check what specifically you need to do
<heat>
i would rather just have a syscall to set the limits of "special shit mate"
<mjg>
but how will you know with onlyo ne branch
<mjg>
or do you wnat to do a search
<heat>
just have a single limit
<mjg>
so only one func which can utilize the machinery?
<heat>
only one library yes
<mjg>
we may be talking past each other here
<mjg>
that is probably a no-go for anything but libc
<heat>
idea being that you would get all these funny functions contiguously using a linker script and then you just set it
<heat>
yes, I care about the libc
<mjg>
i care about the facility being usable in general
<heat>
if the libc wants it can expose these funny functions to users
<mjg>
for example maybe i want a custom malloc to run with my prog
<heat>
it's exactly like the "SIGKILL if syscall in non-syscall page" stuff
<mjg>
and want to be able to use the machinery
<heat>
rseq has a similar problem
<heat>
there's only one struct rseq registered at any given time
<heat>
the libc needs to arbitrate
<mjg>
this is precisley the problem i'm trying to address
<heat>
actually... doesn't rseq do something similar to what we were talking about?
<mjg>
without making the kernel walk any lists to find out if it interrupted a thread within a rseq-esque area
<mjg>
not that i know of, but then again i only skimmed lwn
<heat>
it's shared memory between the kernel and user
<mjg>
well nice shit talking, now i need to do some work
<mjg>
i could easily imagine this being in need for speed 2, 3 or 4
<mjg>
can't comment on the rest
<heat>
no im going to play dark souls
<heat>
bbl
heat has quit [Remote host closed the connection]
FreeFull has joined #osdev
craigo has joined #osdev
Matt|home has joined #osdev
<gog>
praise solaris
<lav>
\:3/
<nikolar>
\o/
<sham1>
/o\
<kof123>
/3:\
<mrvn>
mjg: spinning really depends on the ratio of cores to threads and the time locks are held. E.g. if you have a linked list and one thread per core you always want to spin.
GeDaMo has quit [Quit: That's it, you people have stood in my way long enough! I'm going to clown college!]
Arthuria has quit [Remote host closed the connection]
<immibis>
predicting compiler behaviour is a black art but perhaps just nobody thought to make it do that
<mrvn>
void foo(void) { volatile int x = 0; } Why does it complain x is unused? Initializing it has an affect, that's being used.
<mrvn>
hmm, I was just thinking that I could use a naked function to replace some asm function so I don't have to write the code for ARM and AArch64 twice. Oh how the compiler schooled me: No naked for AArch64 in gcc, no C code in naked functions in clang. unpredicatble stack usage in naked function in gcc / ARM.
<dzwdz>
when would you ever actually want to do that though
<dzwdz>
the unused volatile stack allocated variable
<mrvn>
How am I supposed to know how much stack the compiler will use for some C code?
<mrvn>
dzwdz: probably never.
<dzwdz>
it seems like a pretty useful warning
<dzwdz>
well
<dzwdz>
maybe "pretty useful" is an overstatement
<mrvn>
same warning for static volatile
<dzwdz>
now that's almost realistic
<dzwdz>
does the compiler generate symbols for static variables?
<mrvn>
nothing you could legally use
<mrvn>
Somehow I always end up switching clang to -Os. Why does it unroll loops that much?
<netbsduser>
oh, it's already on the internet archive
<sortie>
^ Been a while since I had a spectacular crazy os screenshot failure
<netbsduser>
i was thinking of finding out how people scan books to commit to it
<heat>
sortie, did you pipe the information to the wrong end? :p
<netbsduser>
my god, the bug has caused a cat to be drawn
<heat>
praise be the UK EU free trade agreement btw
<netbsduser>
does it mean i can buy stuff from europe without having them rummaged through by customs and big bills attached?
<netbsduser>
they were ridiculous, 20%+ VAT
<heat>
i think so yeah
<heat>
"The EU-UK Trade and Cooperation Agreement (TCA)ENโขโขโข provisionally entered into force on 1 May 2021 and provides for zero tariffs and zero quotas on all trade of EU and UK goods that comply with the appropriate rules of origin.
<heat>
"
<netbsduser>
that's good
<heat>
the cheapest I could find on amazon is 17GBP (with shipping already included)
<heat>
which is not horrible
<heat>
apparently also "very good" quality
<heat>
we'll see
<netbsduser>
i was lucky with mine, it looks almost new. you will enjoy it, there's a good ~130 pages on STREAMS
<heat>
fuck yeah
<netbsduser>
very helpful reading for my own lAyErEd iO sYsTeM