<bslsk05>
twitter: <majek04> Are you a student?   Do you like to hack on Linux kernel networking stack?   We're looking for (paid) interns during this summer holidays! You'll do public kernel work, under supervision of our elite Linux team!   (seriously, we're looking for summer interns at Cloudflare)
<heat>
cc gog I guess
<zid`>
okay it *does* work with the stub, just incorrectly, some path seps are gone and no pgoress bar etc
<zid`>
but I can fix that
<zid`>
Program Files (x86)Games best dir name
<heat>
why are you pirating a video game and how does it tie in to a childhood trauma
<zid`>
poverty
<zid`>
to both
<sbalmos>
it's not childhood trauma unless it involved hours of fooling with CONFIG.SYS and HIMEM.SYS
<Jari-->
well first pirate, and then later when you have the means, you buy the games
matrice64 has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<dasabhi>
hello i was wondering, if a shared library is calling a /dev character file that maps to a kernel mod, do multithreaded calls to said driver with ioctl get queueded by default?
<dasabhi>
as in if two threads call fd=open(/dev/w.e) an ioctl(fd)
<dasabhi>
with their own fds
<dasabhi>
is this undefined behaviour land? :S
<dasabhi>
or is it the shared library's job to make sure only one can call the ioctl
<dasabhi>
with some kind of mutex lock
<heat>
dasabhi, that's the kernel's problem
<dasabhi>
ahh okay
<heat>
the ioctl can work completely lockless, or it may need a lock, or it can use a more complex solution (RCU, epoch, etc)
gildasio has quit [Ping timeout: 240 seconds]
<dasabhi>
so the library just needs to have different fds for the different threads, and call it a day
<heat>
no
<zid`>
If you could break the kernel by doing the right sequence of open() calls I'd be very unhappy
<heat>
it can use the same fd
<zid`>
(depending on what that device actually does, that might be completely mandatory, or utterly insane)
<dasabhi>
heat: even if this character driver is sending shit to an actual device for processing for me?
<heat>
yup
<zid`>
A character device would be closer to the "Insane" side, to open multiple times and spam with writes from multiple threads :P
<heat>
see: /dev/tty
<dasabhi>
damn all the threads can share the same fd and bang the same ioctl and its all the kernels problem
<zid`>
but there are very valid cases for that
<heat>
dasabhi, yes
<heat>
it's totally the kernel's problem because it's a security problem
<heat>
a DoS by banging an ioctl from two threads without locking in userspace is stupid
<heat>
the kernel can't *enforce* you're locking, so it can't protect the system from you
<heat>
therefore, it's the kernel's problem
<zid`>
/dev/null not being openable by more than 1 process would be funny though
<heat>
or tty lol
<zid`>
EAGAIN
<dasabhi>
so considering the design of this shared library, I would want to implement a lock and make sure, only one thread can use the fd and call ioctl only one at a time yeah?
<heat>
no
<heat>
again, it's the kernel's problem
<heat>
the only reason why you'd want a lock would be if you want to stop interleaved operations on that fd
<heat>
but that's not something you usually need or want
<dasabhi>
okay that makes a lot of sense
<dasabhi>
ty
<heat>
a problem you may want to tackle in userspace is fd refcounting though
<heat>
sharing fd's between threads isn't that trivial because frequently you can accidentally close() it on one thread
<heat>
if another part of the program opens a new file, and the other thread accidentally writes to it - oops
<heat>
Google made fdsan to try to find those issues
cookie has quit [Quit: off I go~]
<dasabhi>
yeah i am thinking i just make a new fd for each thread that wants to talk to this character device
<dasabhi>
and then make close/delete function in my api for the user to shutdown the specific fd they opened
<dasabhi>
i did consider that issue
<dasabhi>
having one fd means different threads could close it yeah :/
<heat>
have a refcount
<heat>
if the character device isn't seekable/you aren't seeking, that should work
ckie has joined #osdev
<dasabhi>
that works too
xenos1984 has quit [Read error: Connection reset by peer]
xenos1984 has joined #osdev
<geist>
doug16k: did you get your esp32 back up?
<geist>
also, you know, i just noticed you're back. when did you sneak back? you were out for a long time there
<geist>
looks like first time in a while is last saturday?
<doug16k>
yes for almost a year. I just received the first few esp32 today
<doug16k>
correct
pretty_dumm_guy has quit [Quit: WeeChat 3.5]
heat has quit [Ping timeout: 244 seconds]
srjek_ has quit [Ping timeout: 244 seconds]
<doug16k>
binutils killed my project
<doug16k>
must be short for garbagebin utils
<doug16k>
I thought it was binary for a long time
<zid`>
I just banned you from ##binutils
<zid`>
(I didn't really)
<doug16k>
no matter what I did, the first section had 0 VMA
<doug16k>
in uefi
<zid`>
PE BFD is no good? surprising ;)
<doug16k>
some idiotic change for windows to enforce things above 4GB for better address randomization entropy
<zid`>
isn't that vaguely how ASLR always works?
<zid`>
you leave the VMA as 0
<zid`>
and make a shared object
<doug16k>
windows has a "preferred load address" where it is already linked at some address
<doug16k>
and relocations to fix it
<zid`>
yea usually 0x4001000 or something?
<zid`>
one fewer 0 after the 4
<doug16k>
traditionally it was 0x400000 yeah, plus the header offset you mentined, sure
<doug16k>
I realized, I should just stop using the PE linker altogether, and make a little tool that extracts sections from an ELF and make a PE COFF from it
<doug16k>
I made a tool like that ages ago, to make win32 programs from djgpp, before mingw existed
<zid`>
yea I did the same thing when I was hacking a game to shit
<zid`>
then realized I could do it in a much less silly way
opal has quit [Ping timeout: 240 seconds]
<doug16k>
it's extra easy to do it for UEFI, there are no imports or exports
<doug16k>
no resources
<doug16k>
the only part that is nontrivial is the relocations
<kazinsal>
yeah I've had to deal with a bunch of funky DLL preferred load address stuff before
<zid`>
I wanted to link something to an exe permanently
<zid`>
so I wrote a thing to relocate two things together and write out a new binary
<zid`>
which required moving rdata around so that it'd even fit and stuff
<zid`>
then I switched to just patching the entry point to be push "zid.dll"; call main instead, 100x easier :p
<zid`>
(I didn't wanna use an external loader)
<kazinsal>
fun awful fact: GetModuleHandle actually just returns the base address of the DLL you're asking about. that's all an HMODULE is
<zid`>
yea
<zid`>
I've used that fact for nefarious purposes before
<kazinsal>
I have written a lot of brilliantly awful code that fixes bugs in with GetModuleHandle("some_badly_written_game_engine_part.dll") and wrappers for VirtualProtectEx and WriteProcessMemory to inject machine code
<zid`>
Did I ever show you my PATCH ENGINE
<kazinsal>
oh dear
<doug16k>
most fun code injection is CreateRemoteThread with start address at LoadLibrary, VirtualAllocEx memory for a string, WriteProcessMemory the string, set rdi to that address
<zid`>
LoadLibrary happening to match the sig of a new thread so you can use it to inject code's probably the most famous weird quirk evre
<zid`>
kazinsal: That's the dll I inject
<zid`>
I link that to a big array of patches, and a bunch of .o files implementing the patches, gg
<zid`>
it opens an ini to check if the patches should be enabled or not
<dasabhi>
btw when you open a character device file with open i am guessing the r and w macro doesnt matter right?
<zid`>
why wouldn't it?
<zid`>
just because you don't read it in blocks it's not allowed to be read only!?
<zid`>
racism imo
<dasabhi>
:(
<dasabhi>
how exactly do you read and write a kernel module lmao
<kazinsal>
very carefully
<zid`>
what if it's a rom
<kazinsal>
then if it's opened with "w" you get nice and close up to the user's ear so they can feel your breath tickling their cochlea and with the parasocial faux enthusiasm of an asmr youtuber you whisper "undefined behaviour~"
<zid`>
kazinsal: did you like my silly PATCH ENGINE then?
<kazinsal>
yeah it reminds me of some awful things I've done in prod
<zid`>
This is the hyper clean version
<zid`>
compared to huge lists of ifs() and *((int *)0x33341232) = 3;s
<kazinsal>
yeah
<dasabhi>
kazinsal: that was waaaaaay too specific
<dasabhi>
now i need kernel programming asmr
<kazinsal>
that is in fact the method by which I impart wisdom here
<stephe>
hi
zhiayang has quit [Ping timeout: 252 seconds]
gog has quit [Ping timeout: 248 seconds]
k8yun has joined #osdev
Jari-- has joined #osdev
k8yun has quit [Quit: Leaving]
opal has joined #osdev
foudfou has quit [Remote host closed the connection]
foudfou_ has joined #osdev
gxt has quit [Remote host closed the connection]
opal has quit [Remote host closed the connection]
gxt has joined #osdev
the_lanetly_052_ has joined #osdev
opal has joined #osdev
foudfou_ has quit [Remote host closed the connection]
foudfou has joined #osdev
foudfou has quit [Ping timeout: 240 seconds]
foudfou has joined #osdev
the_lanetly_052_ has quit [Ping timeout: 246 seconds]
zaquest has quit [Remote host closed the connection]
zaquest has joined #osdev
nyah has joined #osdev
zhiayang has joined #osdev
sham1_ is now known as sham1
knusbaum has quit [Remote host closed the connection]
Likorn has joined #osdev
knusbaum has joined #osdev
gog has joined #osdev
dasabhi has quit [Quit: Lost terminal]
CYKS has quit [Ping timeout: 260 seconds]
bliminse has quit [Quit: leaving]
gdd has joined #osdev
bliminse has joined #osdev
CYKS has joined #osdev
<Jari-->
hi all
zid` has left #osdev [#osdev]
zid` has joined #osdev
<zid`>
omgomgomgomg
<zid`>
zachtronics' new game is listed on steam and it's.. exactly what I've always wanted
GeDaMo has joined #osdev
<zid`>
engineer of the people remake in not-flash
Burgundy has joined #osdev
xenos1984 has quit [Read error: No route to host]
Burgundy has quit [Ping timeout: 246 seconds]
xenos1984 has joined #osdev
<kazinsal>
zachtronics' new game reminds me that I keep meaning to put together a bbs on the vax but I never get around to doing it
<kazinsal>
could be kinda fun to hook up a sip trunk to it through a cisco box so that people can dial into the bbs
opal has quit [Remote host closed the connection]
opal has joined #osdev
dennis95 has joined #osdev
netbsduser` has quit [Read error: Connection reset by peer]
<bslsk05>
militaryembedded.com: DoD must innovate in AI by 2025 - Military Embedded Systems
<Jari-->
In case if you are interested.
lg has joined #osdev
terminalpusher has joined #osdev
<Jari-->
Today, 100s of billions of dollars in private capital have been invested in 1,000s of AI startups. The U.S.
<Jari-->
I am going to Google this.
blockhead has quit []
muffin has joined #osdev
muffin has quit [Client Quit]
muffin has joined #osdev
Brnocrist has quit [Ping timeout: 276 seconds]
Burgundy has quit [Remote host closed the connection]
gildasio has joined #osdev
heat has joined #osdev
the_lanetly_052_ has joined #osdev
<heat>
don't google
<heat>
bing it
<GeDaMo>
quack quack
<heat>
bing is based
terminalpusher has quit [Remote host closed the connection]
<Jari-->
play some Quacke
<Jari-->
nullmodems anyone?
<Jari-->
"AI ML" jobs, am I with down-to-earth people if I apply?
<Mutabah>
Nowadays, probably
<heat>
you may end up thinking the AI you're building became sentient
<Mutabah>
Just don't make a chatbot, problem solved
<heat>
that's great advice
<heat>
every chatbot becomes racist at some point
<heat>
either that or pseudo-sentient
<heat>
so you're getting fired either way
<Jari-->
Well, and its humans after all that will use it. Meaning you have to know the psychology of your clients. Mathematical equations is not all.
<Jari-->
It would be already an A.I. if you could automate FoxNews.com and make a Pro-Trump news engine.
<Jari-->
But this is where I lost the track.
<Jari-->
I mean, it is fundamentally important, that human will train the system.
<Jari-->
Google has still leftovers, things to do, for me to get my liking news on news.google.com - actually at the beginning it is not very easy, you fail, fail. Etc. But if you add recommendations, like the Google search gives by your searches, it would work, equally as good as news.google.com - with a team of workers, say thousand at least worked on news.google.com ... Don't expect me to do it myself.
<muffin>
I am reading a book about consciousness, it is called "The origin of consciousness in the breakdown of the bicameral mind". oversimplified it says that consciousness is more of a habit than a fundamental thing. sentience though... I think machines are sentient to be honest.
<Jari-->
muffin: I am personally interested in learning more about the human, animal senses. How does it work in mathematical equations excels and so on.
<Jari-->
So how does it make it so quickly available to the person, the senses, all data gatthered, also a todo.
<Jari-->
muffin: sounds your book might be a bit psychological on it's approach. I am looking for very simple equations, explanations I mean, how to sort out, for example web pages.
<muffin>
I have to first understand the variables before I try to calculate it
<Jari-->
Philosophical books lots, practical answers not many.
<Jari-->
muffin: yeah, it is great if you are at marketing
<Jari-->
I understand Google as having multiple senses, it feels something definitely on mathematical, programming level. Imagine if the file disposery, a folder, of images on Google would be human brains. And it sorts out it for searching. Practically I mean when we think we are basically searching the database in our brains. Interconnected with the senses.
<Jari-->
But a man without senses would be really dull company.
<muffin>
well, that's where the book would help you out. because basically it says that one approach to understand consciousness and sentience is to think of it as an epiphonomenon
matrice64 has joined #osdev
pretty_dumm_guy has joined #osdev
<muffin>
s/epiphonomenon/epiphenomenon/
<Jari-->
epiphenomenon? aliasing
<Jari-->
yeah
<Jari-->
its a phenonemon
<muffin>
so, that's a big thing that differenciates us from computers
<Jari-->
yes
<Jari-->
Google might do it for marketing reasons, to study how we spend our money, or what do we like or what do we dislike. But after all, this is also very important for the A.I. development to know, it has pre-rendered sense database available indefinitely, to mimic a human being.
<Jari-->
And its not purely economics it has data on us.
<Jari-->
muffin: wondering how many years until Google can keep up more memory than a 100 year old has even experiences? a sense database.
<Jari-->
Relative amount of practical useful data.
<Jari-->
Mathematically it is larger, but how much pre-rendered data it has, more sophisticated, than our human brains.
<Jari-->
To build an AI, and do the AIML favourite job it seems today. Spam! On employment office.
<Jari-->
muffin: have you noticed how one AIML job for example, has 20, sometimes 30, maybe even 50 jobs positions on subcontractors.
<muffin>
idk man, I never had an interest for office politics
<muffin>
I was always for the focused on the actual work person
<Jari-->
Simply put, semantic analysis is the process of drawing meaning from text. It allows computers to understand and interpret sentences, paragraphs, or whole documents, by analyzing their grammatical structure, and identifying relationships between individual words in a particular context.12 Aug 2020
<Jari-->
This is what I am doing.
foudfou has quit [Ping timeout: 240 seconds]
foudfou has joined #osdev
<Jari-->
Semantics within psychology is the study of how meaning is stored in the mind. Semantic memory is a type of long-term declarative memory that refers to facts or ideas which are not immediately drawn from personal experience.
<Jari-->
Conclusionary?
muffin has quit [Quit: WeeChat 3.5]
<heat>
why is big tech not in your graph
<Jari-->
:)
<heat>
oh
<heat>
these are just startups
ethrl has joined #osdev
ethrl has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
nohit has joined #osdev
matrice64 has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
gorgonical has joined #osdev
mahmutov has joined #osdev
<mrvn>
"Four of the best-funded new AI start-ups — Inflection, Cohere, Adept and Anthropic — have recently poached dozens of AI scientists with backgrounds in Big Tech." You have a startup, you given tons and tons and tons of money and then you expect them to hire stupid people?
k8yun has joined #osdev
demindir1 has joined #osdev
dude12312414 has joined #osdev
ethrl has joined #osdev
Likorn_ has joined #osdev
ethrl has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
dude12312414 has quit [Quit: THE RAM IS TOO DAMN HIGH]
srjek_ has joined #osdev
the_lanetly_052_ has quit [Ping timeout: 246 seconds]
ethrl has joined #osdev
ethrl has quit [Client Quit]
dude12312414 has joined #osdev
dude12312414 has quit [Client Quit]
dennis95 has quit [Quit: Leaving]
smeso has quit [Ping timeout: 276 seconds]
Likorn_ has quit [Quit: WeeChat 3.4.1]
Test_User has joined #osdev
\Test_User has quit [Ping timeout: 248 seconds]
smeso has joined #osdev
mahmutov has quit [Ping timeout: 240 seconds]
Test_User is now known as \Test_User
mahmutov has joined #osdev
sonny has joined #osdev
<sonny>
Are the most recent 'novel' operating systems genode and plan9?
<geist>
guess it depends on your definitino of novel
<sonny>
The more I study bsd, NT, linux the less sure I am of what I would do differently
<sbalmos>
... was about to say the same thing
<j`ey>
fuchsia?
<sbalmos>
capability-based
<sbalmos>
seL4 ?
<sonny>
well, it seems that capabilites don't work in the real world for some reason
<geist>
why do you say that?
<sonny>
seL4 is the verified kernel?
<geist>
i'd like to say the fuchsia kernel, zircon, has a pretty usable capability system
<sonny>
geist: it seems the the fine grain access it gives makes it less convient to work with
<sonny>
ok, I should start looking at fuschia then
<geist>
tis true, also DOS is more 'convenient' than linux, because you can just directly mess with hardware
ripmalware has joined #osdev
<geist>
my point is convenience is fairly subjective, but tends to be a continuum that you have to consider
<sonny>
yeah, looks like finding the right balance is tricky
<sonny>
capability projects for unix don't seem to be popular
<sonny>
what I mean by that is, there's been a few done, but they don't get used
<sbalmos>
"novel" and "popular" don't seem to go together
<sbalmos>
by that definition
<geist>
well that's true, it may just be that posix + capability is fundamentally difficult to pull off cleanly. i haven't thoght about it much
<sonny>
sbalmos this is a different topic
mahmutov has quit [Ping timeout: 276 seconds]
<sonny>
geist well if we consider a file descriptor a capability, then it worked for a while I guess
<sbalmos>
an FD is a handle
<geist>
right, but there's still a fair amount of 'ambient authority' in the posix model
<geist>
like the fact that you can try to open a file, or create a process or allocate memory, or create a thread, etc
<sbalmos>
POSIX "capabilities" are more like the u,g,o+rwxa rights triples, even then it's a stretch
mahmutov has joined #osdev
<geist>
that's what zircon at least tries to do. for the most part (with some unforunate exceptions) you have to have a handle to something to make a new thing
<sonny>
geist yeah, it seems like industry opted to virtualize instead
<sonny>
I do wonder how to make systems that don't require the virtualization hack to protect resources
<sonny>
sharing is the other big deal, computers are everywhere now
<GeDaMo>
What are you protecting them from?
<sonny>
in general 'bad actors', but I guess I could just say processes
<GeDaMo>
Don't allow running processes, problem solved! :P
<sonny>
lol
<sonny>
ultimate system, never fails
<GeDaMo>
There always seems to be some security hole if you allow running arbitrary programs
<geist>
dont have processes: DOS. then run it in a virtual box
<geist>
then dont let it interact with anything
<geist>
also force it to stay in real mode. protected mode is where things get bad
<bslsk05>
arstechnica.com: A new vulnerability in Intel and AMD CPUs lets hackers steal encryption keys | Ars Technica
<GeDaMo>
"Hertzbleed attack targets power-conservation feature found on virtually all modern CPUs"
<geist>
yeah i saw that. >.<
<sonny>
oh, another side channel attack
<mrvn>
If you make a VM per app then the app needs no memroy protection, no capabilities, nothing. The VM though then does.
<mrvn>
GeDaMo: That page won't let me scroll
<GeDaMo>
It works for me but I have JS disabled :P
<sonny>
what a find, these researchers must be very creative
<sonny>
In a distributed OS, it seems that the emphasis is placed on the protocol design, and it's use throughout the system. Is there another major aspect?
<mrvn>
How can that work at all? For there to be any frequency scaling or volatge changes the code has to do different computations for 0 and 1 bits. That's near certain already means a timing attack.
<mrvn>
So at best I would say they found a new way to measure already vulnerable code.
<heat>
apparently not
mahmutov has quit [Ping timeout: 276 seconds]
mahmutov has joined #osdev
ethrl has joined #osdev
ethrl has quit [Client Quit]
<mrvn>
Usulay modern crypto is written to fully compute both cases and only at the last second pick the one actually needed. Is the pipelining going to kill some of the computation when it reaches the final "cmov" and sees one of them gets discarded and thereby change the power requirement?
<mrvn>
both cases of a 0 or 1 bit in a key
<heat>
cloudflare's shit is FIPS certified and even we got fucking pwned
<heat>
it's not a "incompetent devs" problem
<mrvn>
YOu've seen this exploit in the wild?
<heat>
they had a POC
<heat>
you should read the whitepaper
<heat>
they fully extracted a key in 6 hours
<mrvn>
They have something that works in a labatory.
<demindir1>
I think the Hertzbleed issue is much less severe than many seem to think. From what I can tell they only managed to exploit SIKE on a server that does nothing but crypto calculations.
<sbalmos>
gah, head just imploded. took a while for me to understand VMOs go inside VMARs. But why would you have multiple VMARs? And don't start on the whole kernel VDSO thing.
\Test_User has quit [Quit: e]
\Test_User has joined #osdev
<mrvn>
"Vastus Medialis Oblique (VMO) is one of four quadriceps muscles"
<mrvn>
VMAR = Vision Marine Technologies inc?
<sbalmos>
Zircon stuff
gamozo has joined #osdev
<mrvn>
virtual memory address range?
<heat>
sbalmos, the same reason why you can mmap things multiple times
<mrvn>
or array?
<heat>
different regions have different permissions, different sources (VMOs)
<heat>
mrvn, the first iirc
<sbalmos>
yes
<sbalmos>
heat: I have a feeling I'm just getting lost in the terminology
<sbalmos>
(it sucks getting old(er), things not clicking quite as quickly)
<heat>
sbalmos, try doing pmap $$ in your bash shell
gdd has quit [Remote host closed the connection]
<mrvn>
sbalmos: get a piece of paper and a pen and draw out the structure with an example.
<heat>
each entry is a vmar in linux
Burgundy has joined #osdev
<sbalmos>
see I would've thought that you had a single process vmar representing the whole vm address space of that process, and then separate vmos in that vmar for each of those entries
citrons_ is now known as citrons
<geist>
sbalmos: that's precisely how it works
<geist>
you start with a single vmar that represents the entire address space
<geist>
and you can subdivide it with sub-vmars, with different (restrictive) permissions
<geist>
that way you can for example say 'only this sort of stuff goes here'
<geist>
most processes really just have one or two vmars
<heat>
zircon's vmars work like a tree
<sbalmos>
that's what I figured
<heat>
more traditional designs work like an array
<sbalmos>
that's why I was getting hung up was the single all-encompassing vmar by default to a new process
brynet has joined #osdev
<geist>
can help you with that if you need it, being that i designed most of that
<geist>
it's a bit of a conceptual leap but i think straightforward once you get the objects sorted out. it's just more 'basic' than a traditional mmap style scheme
<geist>
ie it's the bits and pieces you can build more complex things out of
<geist>
whcih is in general the strategy of ukernel apis
<heat>
join geist's school of vm design
<heat>
we have mojitos
<sbalmos>
I'm a bourbon guy, but I'll take the mojito
<geist>
re: the VDSO i think that's pretty neat and a cool design we did
<geist>
it has really come in handy too
<sbalmos>
where'd all the good rum go?
<heat>
we're not pirates
<heat>
i'm not opposed to that though
<mrvn>
sbalmos: 99-bottles challenge on leetcode
<sbalmos>
geist: you do sub-vmars to child processes or threads?
<heat>
no
<heat>
each process has its own address space
<heat>
and threads share an address space
<sbalmos>
right...
<heat>
sub-vmars is just a way to partition stuff
<sbalmos>
to itself (the same process)
<geist>
yep. you can say for example create a child vmar that is 0-4GB, and use that for low memory allocations
<heat>
you have [0x0 - 0x7f....], the main div; and then you could have a [0x1000 - 0x30000000] for the main executable, [0x300000000 - 0x500000000] for the heap/mmap, and [0x500000000 - 0x7f....] for stacks and shared objects
<geist>
anything you allocate out of that vmar end sup there
<sbalmos>
sorry, now I was slipping into Genode concept territory, while a parent process tells a child process "you only get 2GB, from 0-4000" at spawn
<heat>
then individual vmars (think mmaps) would partition each partition
<heat>
partitioning all the way down homie
<geist>
yah you can do the same thing in zircon, but it's just done with the vmar scheme: the parent can create the child process, subdivide the vmar, and then only hand the child the sub vmar
<heat>
VMOs are the backend of a VMAR, they let it populate pages when faulting in, do writeback, etc
<geist>
since theres no want to 'mint' a new handle after the fact
<geist>
heat: to be precise VMOs are completely independent of VMARs
<sbalmos>
geist: I think that's where I was mentally going, more classically
<mrvn>
can you add VMARs to threads only?
<mrvn>
thread local VMARs
<heat>
yes, VMOs are independent but serve as the backend for VMARs
<geist>
vmos are simply a bag of bits. you dont have to map them, mapping them is what you use vmars for, but vmos can exist entirely outside of any mapping
<sbalmos>
geist: and then VMOs are essentially a group of kernel-level VM pages?
<geist>
heat: not precisely. again, vmars are the *space* you map a vmo into
<heat>
mrvn, seems conflicting with "per-process address space"
<geist>
a mapping is that, a mapping. it's not a vmar, or a vmo
<geist>
you map inside a vmar
<geist>
it's a common misconception
<heat>
hmm I see
<geist>
sbalmos: basically. it's a bag of data, 0-64bits. tehre are multiple types of them
<heat>
sorry, I failed you master
<sbalmos>
like trying to mentally digest a porterhouse steak when you know the anatomy of a hog
<mrvn>
you don't want to know how sausages are made
<geist>
that's part of the fun of capability and ukernel designs: you try to break down common user patterns into a series of pieces and then give user space the ability to efficiently construct it
<geist>
so you have to remember that in many/most cases average applicatino code should probably not be sitting directly on the syscall api
<geist>
it's okay to build stuff that's intrinsically more fragile or fiddly with the assumption you'll build some sort of user space library or whatnot to encapsulate that
<mrvn>
.oO(C code has text, heap and stack. Just give userspace 4GB of address space and dynamically map pages on demand)
<geist>
sure, you can do that in zircon easily (though you could in linux or whatnot too)
<heat>
that was my first VM design
<heat>
map any page when faulting
<heat>
well, design is a strong word :P
<mrvn>
my first design just mapped a few pages at the start and hoped it's enough
<heat>
actually, no, that was my first vm design with userspace
<heat>
my *first* vm design used large pages for everything
<heat>
I also had a horrible, buggy heap. each bucket allocated 4MB for itself
<mrvn>
you mean you didn't map individual obejcts but just had a big address space?
sonny has joined #osdev
<heat>
no objects here
<heat>
just plain vm_map(address, length, perms)
<heat>
oh yeah ofc no address allocation as well
<heat>
hardcoded large pages >> linux
<mrvn>
So allocation a struct Task would use 2MB?
<heat>
no
<heat>
the size bucket would use 4MB
<heat>
if you ran out of memory... tough shit
<heat>
ran out of memory in the size bucket, that is
<bslsk05>
github.com: Onyx/heap.c at 9d6c0cc7f876ae06687eadd1d0d076b2ee50fce1 · heatd/Onyx · GitHub
<mrvn>
My memory management is super simple. phyiscal pages are in a stack, each process (and the kernel) has a PageTable with an iterator pointing at the next palce to map pages. You allocate something then it maps pages and increments the iterator by one more. If it hits the end of the PageTable it panic()s.
<heat>
worth noting that this was GPLv2 code lol
<heat>
also soooo bad
<heat>
embarassingly bad almost
<heat>
very fun to see my evolution in a single git repo
<mrvn>
I should really implement the warp around for the iterator.
Brnocrist has joined #osdev
<alethkit>
Hi there! What are your thoughts on exokernels?
<mrvn>
yes
<alethkit>
Yes?
<mrvn>
bunnies?
<mrvn>
invisible pink unicorns
<mrvn>
What are your thouhgs on cars?
<sonny>
exokernels is what allows for the library OS?
<mrvn>
it's DOS in a library :)
<alethkit>
sonny: Pretty much
<sonny>
seems really tedius unless you have a special need for it, like long running server apps that need as little overhead as possible
<sonny>
or maybe deeply embedded stuff
<mrvn>
or spooling up apps as VM instances
<sonny>
yeah
<alethkit>
Spooling?
<mrvn>
booting
<alethkit>
Ah, right.
<alethkit>
I was trying to figure out a way to get around the driver chasm, and exokernels seemed like the best idea
<heat>
not feasible if you need security
<alethkit>
Why not?
<heat>
driver chasm?
<heat>
it's a library. your rando not-virus.exe can see everything that's going on
<sonny>
alethkit I'm pretty sure exokernel is only popular for networking
<alethkit>
heat: Driver chasm refers to the fact that no new hobby/research OSes can take off because writing drivers is a PITA
<heat>
rumpkernel
<alethkit>
And I thought that the libOSes were isolated?
<heat>
also, nothing really takes off because there's no advantage
<heat>
internal APIs are usually very different. using the bare API gives you some big advantages
<alethkit>
The bare API being the kernel's userspace API?
<heat>
no, the kernel's internal API
<mrvn>
alethkit: you only need to support a handfull or virtio devices.
<sbalmos>
geist: okay, mind is starting to solidify... but if a vmo is the representation of a extent of kernel vm pages, and a process has to map it into a vmar in order for the vmo to be part of its per-process address space... what's the practical use of a vmo that isn't mapped? The only thing that could read/write from an unmapped vmo would be the kernel then, right?
pretty_dumm_guy has quit [Ping timeout: 258 seconds]
pretty_d1 has joined #osdev
<heat>
bag of bytes you can read/write and pass to other processes
<heat>
like a temporary file
sonny has quit [Ping timeout: 252 seconds]
<heat>
it's pretty much an inode in a filesystem sense
<sbalmos>
right I know. But it's just the object level of "here's 4k of memory that is writable". But the vmar mapping is what says that 4k exists at process address space 0x123456
<heat>
an unnamed inode I mean. has data, can be mapped, will go away after closing
<alethkit>
mrvn: I guess that makes sense for a hobby OS, but I have no idea what to do now
<alethkit>
My intention was to get the exokernel to load on an old computer so I could have multiple OSes knocking about on bare metal
<sbalmos>
so unless it's vmar mapped, it can't be used by a process because it doesn't exist in the process's address space, and therefore the only thing left (e.g. not a userspace process) which could read/write it is the kernel
<heat>
sbalmos, it has syscalls attached to it
<heat>
read, write, etc
<heat>
you don't need to map it, but you can
<heat>
just like a file
<sbalmos>
maybe that's the next conceptual leap. Process requests 4k to write to, it gets a VMO. It doesn't/has no way to know where that 4k lives in its per-process address space, but it can do whatever with it. like an fd/inode. It could then mmap that to a vmar for whatever purpose if desired.
<heat>
VMOs don't live anywhere in the per-process address space until they're mapped
<heat>
they're only present as a physical page + a physmap mapping as usual
<sbalmos>
right...
FreeFull has joined #osdev
<geist>
right, the closest analogy to a vmo handle is a handle to a deleted file in a tmpfs in posix
<geist>
ie, you can read/write it directly, and infact it's precisely the basis for memory mapping files and whatnot
<geist>
and the actual read/write syscalls are the mechanism by which you cna read/write files in a remote FS server, which vends VMOs as part of its api
<geist>
but vmos also form the basis behind anonymous memory, much like mmap("/dev/zero") in that sense
<geist>
vmos are also snapshottable (create private copy on write clones of it) so you also get the mechanism to do mmap(..., PRIV)
sonny has joined #osdev
<sbalmos>
mentally I was just going for this middle space below a typical userspace program possibly requesting from the allocator "gimme 4k of ram, preferably from this previously-defined address space". It's a little lower level where the memory range itself could be separately r/w as an opaque "file" (bad relation, I know), before traditionally becoming an actual per-process address space range
<sbalmos>
same idea for shared mem i/o
<geist>
yah also shared mem with vmos is intrinsic: you simply give another process a vmo handle
<sbalmos>
right
<geist>
then it is free to access the same thing. there's no separate mechanism for it
<sbalmos>
though then my head starts floating to processes using (untyped) shmem rather than (typed) channels, etc
<mrvn>
do VMOs have a reference count?
<geist>
yes
<geist>
all objects do. all kernel objects are removed when the last ref goes
<geist>
if you mean: can user space read the ref count, no
<geist>
that would at best be a TOCTOU problem anyway
<sbalmos>
next dumb question, any reason why the syscall prototypes all work on the base zx_handle_t type, rather than say having a zx_vmo_handle_t, zx_proc_handle_t, etc subtypes?
<heat>
it's just a number
<heat>
exactly like a fd
<heat>
(but random?)
<heat>
it's not typed
<geist>
that's right. there's a single handle table, all objects are in the same table
<geist>
and some syscalls operate on all handles, like get info, etc
<geist>
and clone handle, close handle, etc
<sbalmos>
right, but it's wrong to say they're not typed when the syscalls all say "handle must be type ZX_OBJ_TYPE_PROCESS" etc
<sbalmos>
I get that at that point it's a type field on the handle struct.
<heat>
but C isn't strongly-typed
<sbalmos>
ah, see there's the dumb-idiot kicker. this is C level, not C++?
<heat>
typedef uint32_t zx_proc_handle_t and typedef uint32_t zx_vmo_handle_t will both implicitly cast from one to the other
<heat>
C
<heat>
(doesn't C++ also do this? I believe so)
<heat>
you /could/ try to get around it with fancy C++ though, but at the end of the day the syscall gets a uint32_t
<heat>
I believe there are C++ bindings for zircon somewhere in the codebase
<sbalmos>
nevermind
<j`ey>
I thought zircon was C++?
<sbalmos>
no idea why, but stuff like that bugs the frickin' heck out of me
<heat>
i meant for the syscall layer
<heat>
you have the usual C bindings and then some C++ stuff as well
gdd has joined #osdev
<sbalmos>
seemed like there would've been a more typesafe way of differentiating handle types
sonny has quit [Quit: Client closed]
<heat>
in C++ there probably is, not in C though as far as I can see
<j`ey>
you still have to convert from a raw u32 at some point, since it just comes from a register
Bonstra has joined #osdev
<heat>
unless zx_handle_t's become structs? in which case things may start to go bonkers in codegen
<heat>
safe to say that all these build systems are more sane than autotools
<heat>
mjg_, hm?
<mjg_>
> 00:13 < heat> more sane than edk2's still
<heat>
yeah lol
k8yun has quit [Quit: Leaving]
<heat>
looking at edk2's build system?
<heat>
everything is overridable in 10 different places
<heat>
also *everything is a fucking GUID*
janemba has joined #osdev
ethrl has joined #osdev
ethrl has quit [Client Quit]
<sbalmos>
a sha hash is better? :D
<gamozo>
2010s devs love GUIDs
xenos1984 has quit [Read error: Connection reset by peer]
<heat>
gamozo, this was designed in the 2000's microsoft GUID kool aid era
<heat>
so obviously everything needs to have a GUID
<heat>
my co-mentor (long time Intel FW engineer) in the edk2 GSoC has a vscode extension to generate GUIDs
SGautam has quit [Quit: Connection closed for inactivity]
<gamozo>
Oh yeah... I forgot about the whole PE side
<doug16k>
they are effective as scarecrows to scare people away. who wants to edit "{a1bc23cb-3456-bcde-abcd-feb363cacc88}"
<heat>
wintel people
<gamozo>
(although, I still like PE more than ELF!)
<heat>
booo
<gamozo>
They have their pros and cons!
<heat>
boooooooooooooo
<heat>
UNIX is kool
<gamozo>
DWARF is a terrible format and should be illegal
<heat>
so a.out or ELF
<heat>
gamozo: MOFO WHERE U LIVE MF I WILL BEAT U UP
<gamozo>
so harsh
<heat>
dwarf iz best 4mat
<heat>
pdb bad
FreeFull has quit []
xenos1984 has joined #osdev
nyah has quit [Quit: leaving]
<sbalmos>
heh
<sbalmos>
I was on the CLR-kernel bandwagon my senior year back in '06. Groaned at PE but whatever.
<sbalmos>
that was fun, from an on-paper conceptual standpoint
<geist>
re: type safety handles and whatnot
<geist>
yes, there are language specific libraries on top of the syscall layer that do precisely that
<geist>
libzx for c++ iirc, and there's a rust version of it
<geist>
it encapsulates a bunch of that logic, though of course you're free to force one handle type to another
<geist>
and the kernel will obviously test. but the raw api for the syscall is C, because that's the base abi that you can actually nail down now and forever more and build bindings for other languages
<geist>
that's why the VDSO itself exports C functions
<sbalmos>
ya I figured the higher-level bindings did that
<geist>
there *is* a syscall to ask what kind of handle this is, or if the handle is valid, etc
<geist>
so you can if given a random handle, validate that it's what you think it is
Likorn_ has quit [Quit: WeeChat 3.4.1]
<doug16k>
do you have a mechnism that tries not to give a process the same handle twice, in case of use after close?
<sbalmos>
I know this is probably now melding the vx_* syscall layer vs the userspace bindings libs... is there a capability that has to be granted or something for a process to allocate from heap? otherwise what prevents a process from calling vx_vmo_create(BIGASSSIZE, ...) or being restricted to some amount of total vm granted by the parent