<dinkelhacker_>
I'm reading this old paper about multi threaded userspace servers for os services in microkernels. They say the advantage of the microkernel is that they can have multiple threads in the servers as opposed to Unix that according to them is single threaded. If unix had kernel stacks per thread, the os services would have also been multithreaded,
<dinkelhacker_>
right? I mean at least the parts that are lock free.
<heat_>
are these lock free parts in the room with us?
<dinkelhacker_>
?
<heat_>
UNIX was big-kernel-locked for a loooooooooooooong time
asarandi has quit [Quit: WeeChat 4.2.2]
<heat_>
openbsd still is in many ways
<dinkelhacker_>
Like what does that mean in practice? They locked/unlocked on kernel entry/exit?
<heat_>
yes, or on kernel sleep
<heat_>
also, i dont see what you're quoting
<heat_>
i see "as opposed to the single thread of control in conventional Unix operating systems" in page 2, which means something completely different
<heat_>
unix at the time had no threads interface
heat_ is now known as heat
<heat>
if you wanted a thread, you fork()d
Left_Turn has joined #osdev
asarandi has joined #osdev
<dinkelhacker_>
But if those threads had separate kernel stacks they could both make process inside the kernel using the same service.I mean at least theoretically. Maybe not in practice because of the BKL issue you mentioned.
<heat>
but it's not talking about the kernel
<heat>
"Mach also supports the concept of multithreading" comes right before it
<heat>
UNIX did not have threads, it was not a thing, you fork()'d if you wanted concurrency
<heat>
that's what it's talking about
<heat>
note that the whole paper is bullshit, you do not start a thread for every request if you want to be in the remote ballpark of "fast"
<heat>
it would be mindnumbingly slow to create and destroy a thread for every request you get
<dinkelhacker_>
Hmm maybe I understood it wrong but I thought it compares the OS Service that Unix would run inside the kernel with the microkernel approach that would run the same Service as a userspace server.
<heat>
oh wow this is actually teeeeeeeeeeerrible
<heat>
i refuse to believe this was good in 1996
<heat>
in the multi_threaded_server, they get a request, lock buffer_lock, create a thread. and then the created thread unlocks buffer_lock and does some IO, then exits
<dinkelhacker_>
I haven't even thought about if its good or bad. I just wondered if the mentioned argument is even true.
<heat>
not only do you create a thread per request, you also need to wait for the spawned thread to run for a bit before handling another request
<heat>
asynchronous IO with 50 extra steps
<heat>
it is true in the sense that they're talking about how the shiny mach has this novel thing called "multithreading" where UNIX used processes for everything
<heat>
both user and kernel threads did not exist there
<heat>
in fact linux and plan9 use processes-but-they-share-a-lot-of-shit as threads
<heat>
and (this is where my knowledge gets fuzzy and imprecise) the BSDs used lwps which AIUI are kind of specializations of the everything-is-a-process-that-may-or-may-not-share-a-bunch-of-shit
<froggey>
heat: back in the 90s webservers would fork() for every request
<heat>
they still do
<froggey>
yeah
<heat>
well, they fork, but not for every request, usually
<froggey>
no, it was for every request
<froggey>
shit was bad
<heat>
yes, im talking about now
<froggey>
I'm talking about the 90s
<heat>
i'm aware it was a thing for webservers, but I would expect this to ultimately be super shit for e.g an IO server
<heat>
where you're bound to get more, uncontrollable traffic
* Ermine
. o O (tipidee)
<heat>
tipidee forks for every request?
<dinkelhacker_>
heat: But does it matter? So a process did have a kernel stack, right? So each client of the OS Service did have it's own thread inside the kernel and could independently make process.
<Ermine>
heat: it works under superserver, so naturally
<heat>
ew.
<heat>
that's actually bad
<heat>
i know of a certain proprietary web server that also does that and led to _many_ operational issues in both userspace and kernel space once you reach a certain level of traffic
<heat>
no one is prepared for 2000 threads sharing some CPU cores
<heat>
dinkelhacker_, yes?
<heat>
dude they're just having a wank thinking about how they have this cool multithreading thing that the UNIX losers dont
<Ermine>
when i get this much thraffic i'll switch to nginx
<heat>
having fucking multithreading was like having eBPF
trillion_exabyte has quit [Ping timeout: 260 seconds]
<heat>
super far fetched high tech stuff
<heat>
you need to read that sentence like it was written 30 years ago, because it actually was written 29 years ago
<dinkelhacker_>
heat: I know :D .. I just found it weird comparing that because the Unix in-kernel version would also be "multi-threaded" because each client has its own thread inside the kernel.
trillion_exabyte has joined #osdev
kof673 has joined #osdev
<Ermine>
anyway, that's all about "how to make microkernels fast"
Turn_Left has joined #osdev
<heat>
win32 CreateThread seems to have appeared in windows 95
<heat>
one year before that paper
<Ermine>
windows 95 is the most advanced OS out there
<heat>
obviously, it has threads!
dinkelhacker_ has quit [Quit: Client closed]
Left_Turn has quit [Ping timeout: 265 seconds]
<zid>
heat I am making nikolar
<zid>
write locks
<zid>
PESSIMAL locks
<nikolar>
the spinning kind!
<heat>
wdym pessimal
<zid>
I don't think anybody ever found out
<zid>
PESSIMAL just is a thing you write in all caps
<zid>
when you think they could maybe be better?
<heat>
my locks also spin
<zid>
did you do the fancy 'try take it without LOCK, if it fucks up, try take it with lock cmpxchg' thign?
<zid>
you were benching locks for fucking years with.. thingy, geezer
<Ermine>
funkytown kicks in
<heat>
>try take it without LOCK
<heat>
you can't do that
<zid>
there was some thingy
<zid>
where you did it less robustly one time and if it was already taken you span or whatever
<nikolar>
you can actually
<nikolar>
i fogor the details
<heat>
so, what you want for OPTIMALITY is: a fastpath consisting of "try to take the lock, if not spin". spin (aka slow path) consists of *reading* the lock word with no atomics, waiting for the lock to be free. when it is, attempt the lock cmpxchg (or xchg) again. if it fails, go back to reading
<heat>
this is for a test-and-test-and-set lock
<nikolar>
yeah that's what he was referring to
<zid>
so yes
<zid>
yes you do
<Ermine>
if you're lucky, everything goes in the right order and you don't need locks!
<heat>
no, you can't "take it" without LOCK, you can only spin without lock
<zid>
try take, if not, speen
<nikolar>
well anyway
<nikolar>
we're not doing that :P
<heat>
shame, shame on the house of zid and nikolar
<zid>
do you PAUSE
<nikolar>
oh good idea
<nikolar>
i forgor
beto has quit [Remote host closed the connection]
[Kalisto] has joined #osdev
beto has joined #osdev
<heat>
if you're fancy and like bad ideas: mwait locks
<nikolar>
there
<nikolar>
now i pause
<zid>
Can we use BOUND somehow
<zid>
and all the other insturctions you never see
<nikolar>
and the XLAT
<nikolar>
was that what it's called
<nikolar>
the lookup thing kek
<zid>
ooh yea
<heat>
AAA
<GeDaMo>
pshufb can be used for small lookup tables :P
brynet has quit [Remote host closed the connection]
<kof673>
x is sometimes used for trans, hence translate https://www.seventhstring.com/xscribe/overview.html # Transcribe! is the world's leading software for helping musicians to work out music from recordings # XScribe™ Cardiac Stress Testing System # the last one is possibly a misunderstanding :D
<bslsk05>
www.seventhstring.com: Transcribe! - software to help transcribe recorded music
<kof673>
xlat is just "abbreviated"
<kof673>
mind: > From the Latin mens, a fact unknown to that honest shoe-seller, who, observing that his learned competitor over the way had displayed the motto "Mens conscia recti," emblazoned his own front with the words "Men's, women's and children's conscia recti."
Turn_Left has quit [Ping timeout: 252 seconds]
Left_Turn has joined #osdev
joe9 has joined #osdev
surabax has joined #osdev
Turn_Left has joined #osdev
heat_ has joined #osdev
Left_Turn has quit [Ping timeout: 248 seconds]
heat has quit [Read error: Connection reset by peer]
imyxh has quit [Quit: WeeChat 4.4.3]
<Ermine>
re kgdb, i guess it doesn't work on network?
<nikolar>
kof673: mind is a germanic word
<nikolar>
unrelated to latin (other than through pie)
<kof673>
no contest, was just saying misunderstandings happen :D
<kof673>
it is the linguistic version of cargo cult
<zid>
I have
<zid>
mini sausage rolls
<nikolar>
congrats
<zid>
Envy is free, so knock yourself out
joe9 has quit [Quit: leaving]
heat_ has quit [Read error: Connection reset by peer]
heat_ has joined #osdev
jcoder has joined #osdev
<jcoder>
Hello
<GeDaMo>
Hi jcoder :)
<jcoder>
I'm getting a weird error when compiling Meaty Skeleton - kernel/kernel.o: file not found
<jcoder>
stdio.h
<zid>
okay
<Ermine>
do you have stdio.h ?
<zid>
You might wanna like, make a pastebin with enough detail for someone to actually help you
<jcoder>
Yes, in sysroot/usr/include
<zid>
what your commands are, what your filessytem looks like, etc
<jcoder>
...
<Ermine>
why would you use userspace header to build kernel
<jcoder>
Oh wait, it's now "libk.a: error adding symbols: file format not recognized"
<zid>
trying to build *what* though. if you're trying to build the kernel then that's a catastrophic path failure, including the wrong objects into each other
gog has joined #osdev
<jcoder>
I
<zid>
If you're trying to build something for sysroot/bin it means libk.a is built incorrectly
<jcoder>
'm trying to compile libk.a, in sysroot/usr/lib
<zid>
then you can run file on the .o files and check that they're all ELF 32-bit
<jcoder>
all 64-bit
<jcoder>
is that the problem?
<zid>
yes
<jcoder>
ah, since we're targeting i686 instead of x64
<zid>
you can't link .o files for different CPUs together
msv has joined #osdev
<jcoder>
right. learned that from scouring osdev wiki forums to figure out why my crtbegin.o wasn't freaking linkind :-O
<zid>
your i686-gcc's binutils probably can't even read them
<jcoder>
linking
<jcoder>
right
<zid>
hence, "File format not recognized"
<zid>
which is not a very useful error, but this explains why
<jcoder>
right... how can I fix this?
<zid>
you gave an ELF64 to a 32bit binutils and it said "what the fuck is this"
<zid>
don't build libk.a for 64bit?
<zid>
maybe you accidentally used your system gcc
<jcoder>
ok...
<zid>
instead of that i686 one you have
<jcoder>
got it. I used the one from the original meatyskeleton and copy-pasted it, make my mods to it, and this error happened
<jcoder>
lemme look in the Makefile
<jcoder>
nothing
<jcoder>
hello
<jcoder>
?
jcoder has quit [Quit: Konversation terminated!]
<zid>
I don't understand people who think *that* is how you communicate? Does he walk into a pharmacist for his broken leg, walk up to the cashier and stay "Nothing" to them, then just stand there until he gets bored and leaves?
* gog
hobbles in'
<gog>
hello??
<gog>
there'es something wrong with my leg
<zid>
any symptoms?
<gog>
it hurts and there's a bone sticking out of it
<zid>
There's suppsoed to be a bone in your leg, dw
<gog>
oh
<kof673>
behaviorism requires ........feedback and imitation
<kof673>
the reverse, is a crowd just ignores everything you say......... it is for pigeons and dogs
<nikolar>
Kek
xenos1984 has quit [Read error: Connection reset by peer]
<kof673>
"SKINNER!" skinner: superintendant Chalmers! # for he at's 90's education the frogurt is also cursed!
jcoder has joined #osdev
jcoder has quit [Quit: Konversation terminated!]
xenos1984 has joined #osdev
goliath has quit [Quit: SIGSEGV]
craigo has joined #osdev
<kof673>
budget make, only compile if needed (then i will add optionally fail if CC did not create the fail): $FILE_EXISTS ${OBJ_DIR}lnpal${OBJ_SUFFIX} || $CC $CPPFLAGS $CFLAGS $OUTPUT_FILE_FLAG ${OBJ_DIR}lnpal${OBJ_SUFFIX} $COMPILE_TO_OBJ_FLAG lnpal.c
<kof673>
*create the file.
<kof673>
that will at least not rebuild if already built, but of course does not look at dates/etc. or have any concept of a file being modified needing other files to be rebuilt
<kof673>
however, an external tool that did that, could just remove the particular objects to force rebuild of a particular portion :D
<bslsk05>
github.com: Free95/free95/src/kernel.c at main · Versoft-Software/Free95 · GitHub
<ring0_starr>
the author posted it on leddit r/osdev
<mpetch>
leddit for luddites
<ring0_starr>
do tutorials and meaty skeletons do more harm than good?? you enable somebody with this level of skill in coding to perform some systems level programming
<ring0_starr>
by the time i started my OS, i had already built several mid sized C++ applications
<ring0_starr>
and i was like beginner-intermediate level in reverse engineeering
<kof673>
putchar() could use a switch, is that what you are referring to :D
<mpetch>
The inline assembly has some subtle bugs too
<ring0_starr>
what i'm referring to is the highly uneven nature of skill
hwpplayer1 has joined #osdev
<kof673>
and if it is sequential, probably doesn't need index = x * 16 lines either, just calcualte x once, and multiply once
<ring0_starr>
somebody who doesn't know switch statements exist, or a better way to do what's being done in that function, shouldn't be thinking about inline assembly at all
<kof673>
i'm ok with amateur stuff, buggy code, on the theory that separate functions/isolated...i can theoretically revise just that one function.... design issues are more serious, or if the API is bad
<ring0_starr>
you can't separate concerns like that
<kof673>
i don't know why somethings are INT somethinsg are VOID others are not. i might do that, but a new name, and within a "module" it would presumably all do that, not mix and match
<ring0_starr>
interface design coming from a person that wrote a single function like that isn't going to be good
<ring0_starr>
there is no separation of concerns by the way
<kof673>
sure, i just mean is the bad code isolated or not?
goliath has joined #osdev
<kof673>
it is vaguely oo/encapsulation/black box........i don't care what it does internally, just it either works or not (for some definition of "works")
<ring0_starr>
that was CS-graduate-meme-tier code for sure, but the rest of it isn't great either
<zid>
speaking of subtle bugs, we're trying to get code working on nikolar's laptop
<zid>
it is being very uncooperative
<zid>
dying somewhere really early in the kernel code, but runs all the bootstrap code flawleslly no matter how rough we are with it :P
<kof673>
KiDrawBitmap mouseCursorBitmap i'm not sure i follow the naming style :D
<mpetch>
A lot of the code is from James Molloy's tutorial including a number of the well known bugs
<kof673>
snake_case_for_me CONSTANTS this is...i don't know :D
<kof673>
*CONSTANTS and DEFINES
<ring0_starr>
it's like he had windows NT inspiration
<ring0_starr>
but then regressed back to java
<ring0_starr>
<zid> it is being very uncooperative hmmm speaking of uncooperative
<ring0_starr>
is there any OS that does semi-cooperative multiprocessing? like scheduled code marks where it would like to pause, but instead of being a yield syscall that gets executed, the scheduler reads these execution breaks directly and uses it to inform its scheduling
<ring0_starr>
a scheduler is most efficient when it has perfect information about how long each task will take, in advance
<heat_>
define "inform its scheduling"
<zid>
heya peat
<heat_>
hello
<ring0_starr>
like perhaps the compiler says "i estimate this section of code will take XX number of cycles" or "i don't know how long this will take, but probably greater than a quantum", etc.
<zid>
kernel says triplefault
demindiro has joined #osdev
<demindiro>
ring0_starr: sched_yield
<ring0_starr>
that takes no arguments
<kof673>
i have thought of things like that, will likely never get there....profiling, feedback, jit.......just keep lots of stats..... isn't that somewhat the branch "likely" "unlikely" does on a miniature scale for the cpu :D
<ring0_starr>
kof yeah but this would influence when the task gets scheduled
<demindiro>
It suspends immediately. Counting cycles is extra work for no real reason since either it is: 1) correct, in which case you might as well just have cooperative scheduling and not bother or 2) incorrect, in which case you want to preempt anyhow
<ring0_starr>
inter-priority
<kof673>
likely/unlikely of course is not saying how long something takes, just "how often you expect it to be reached" perhaps
<kof673>
i mean...annotate code with an estimate of time?
<kof673>
#pragma to the rescue j/k
<ring0_starr>
basically?
<ring0_starr>
could be achieved with some extension to PGO
<kof673>
yeah, that is what i thought of
<ring0_starr>
but the idea is to try bringing scheduler efficiency closer to theoretical ideal
<ring0_starr>
demindro says it's futile to attempt this
<demindiro>
Not futile, just pointless
<ring0_starr>
but without some information you basically have to take dumb approaches
<kof673>
:d i think a real time system, with limited controlled data, it makes sense. general purpose, arbitrary processes, etc. the only hope is enough statistics would eventually "converge" on a "typical workload"
<kof673>
could do more harm then good
<ring0_starr>
this wouldn't actually make much sense on a real time system.
<kof673>
nevermind overhead
<kof673>
i mean data flow, say arguments to a function, are going to vary...
<ring0_starr>
the idea started out in that context but wouldn't really work because real time is all about guarantees
<kof673>
wc -l one_line_file versus wc -l <300M file>
<kof673>
how long does wc command take?
<kof673>
depends on the data
<ring0_starr>
this would be more for like server workloads
<ring0_starr>
yea of course there'd be a marker for "computation time varies based on external input"
<ring0_starr>
which basically would put the task into a class that cannot be optimized
<kof673>
> C-Mix is a partial evaluator that is able to specialize strictly conforming ISO C programs. C-Mix generates specialized versions of functions
<heat_>
the preempt count is kind-of that
<heat_>
and linux may be gaining user-space preempt count support Soon(tm)
<kof673>
i just wanted something hooked to that, so it would be helpful to determine the......heated spots......that should be specialized
<kof673>
totally different than hot spots /s
<heat_>
the problem is that no scheduling system is ever smart enough to ever take cycles estimations or time estimations into account more than "huh we're going to be here for a little while? then avoid scheduling out"
<zid>
That's why you should spend 200ms every 10ms asking chatgpt
lanodan has quit [Ping timeout: 272 seconds]
m5_ has quit [Ping timeout: 268 seconds]
<ring0_starr>
heat_: basically my question was, is there a scheduling system that uses estimates to split tasks into classes and then sorts them within subclasses
<ring0_starr>
the goal is to maximize throughput of tasks
<ring0_starr>
so you'd want to order them in shortest first to longest last, accounting for priority somehow too
<heat_>
no
cloudowind has joined #osdev
<cloudowind>
goodays osdev people how are we doing
<zid>
"Chatgpt, what task should I switch to? answer quick pls I have 0.1ms to respond"
<demindiro>
outoftime-killer?
<demindiro>
Easy fix
<demindiro>
(/jk)
<zid>
It told me to drink my own pee
<mpetch>
filtered or unfiltered pee?
<zid>
do kidneys count
<mpetch>
lol
<heat_>
chatgpt diy dialysis tutorial
<cloudowind>
i know someone drinking his own pee everymorning here in australia his name is trever cocks
<cloudowind>
a good friend of mine , told him not to do but he thinks it something holy and healthy crazy person you guys have a good one
cloudowind has left #osdev [#osdev]
<heat_>
trever cocks
<zid>
I am waiting for nikolar to very very slowly do small edits to the memory mapping code until his laptop and qemu are both happy with their respective E820s etc
<kof673>
"salt" supposedly absorbs sun and moonlight...some of these things may be ...misinterpreted ancient stuff :D
netbsduser has joined #osdev
demindiro has quit [Quit: Client closed]
<kof673>
it would be nice to annotate code with the O(n) stuff.......then other code can look at available algorithms, and decide which to use. minor thing, just having this explicit and pervasive
hwpplayer1 has quit [Remote host closed the connection]
<kof673>
no just in general
<kof673>
reflection in a way, being able to query that about a function in a standardize way, code is marked up/tagged such, so runtime decisions can use that if desired
<kof673>
which i assume you would want the timing information like that :D
Gordinator has joined #osdev
msv has quit [Remote host closed the connection]
msv has joined #osdev
<ring0_starr>
well more than that, in my use case it doesn't really matter since you still need to know the N, and the implementation too
<ring0_starr>
cycle counting is more concrete and gives the scheduler something easy to work with to make decisions
<ring0_starr>
oh yeah, and syscalls pretty much ruin this setup
hwpplayer1 has joined #osdev
netbsduser has quit [Ping timeout: 268 seconds]
GeDaMo has quit [Quit: 0wt 0f v0w3ls.]
<kof673>
if you really wanted to be funny, how does power use for code using certain instructions versus others? like choosing code based on cpu, a toggle switch at run-time for "low power mode" that uses the "low power" functions lol
TkTech has joined #osdev
m5_ has joined #osdev
hwpplayer1 has quit [Quit: sleep]
dormito has joined #osdev
Left_Turn has joined #osdev
<ring0_starr>
iunno maybe
Turn_Left has quit [Ping timeout: 272 seconds]
<ring0_starr>
there's libcs that overwrite a trampoline to basic string functions based on cpu feature set
<heat_>
ifunc does not overwrite any code
<heat_>
ifunc only touches the GOT
<ring0_starr>
i guess
<araxestroy>
overwriting code is the domain of heretics like me
<ring0_starr>
i could've sworn i recalled seeing what i guess is the plt get overwritten for SSE support
<ring0_starr>
anyway that's beside the point. the point is, power efficient instructions would be what, regular ones so that the vector unit could be put into a low power state somehow?
<ring0_starr>
not that current cpus can do that
<ring0_starr>
shutting off power to a large part of the cpu die though sounds like it might do something
xenos1984 has quit [Read error: Connection reset by peer]
<heat_>
araxestroy, there's a similar-minded heretic in linux trying to do that
<heat_>
something like this is crucial in order to share .text when patching
<ring0_starr>
linux already does this with functions marked as init
<ring0_starr>
unloads themi mean
<ring0_starr>
learned about this feature the hard way lol
xenos1984 has joined #osdev
steelswords94361 has joined #osdev
<kof673>
yes, i'm not sure it breaks down so simply, from stack overflow, i was thinking that: > SIMD is where some instructions really start to cost significantly more energy i am just assuming you'd give a compiler a magic switch to at least avoid certain things, etc. but then of course, does your avoidance even save anything, or just give slower code, that in total uses just as much or even more power?
<kof673>
now, if a cpu was specifically designed that way to give you alternatives...
Mutabah has quit [Remote host closed the connection]