<bslsk05>
en.wikipedia.org: Tailored Access Operations - Wikipedia
<heat>
\Test_User, nah, it runs on every CF server, on-demand
<heat>
javascript/WASM is actually perfect for this
<heat>
since the startup time is minimal
<epony>
for malware
<heat>
doing this with containers or vms would not work
<epony>
since you can't tear it out once your applications "demand" it is needed
<epony>
it works
<epony>
programs like hypervisors and emulators are not immune to gadget construction and needle run through their large stacks of data for constructing and takeover of the execution flow
<epony>
they also run on the same real CPU sharing resources, like networking, caches and lookaside table and stack pointers and buffers and what not
<epony>
virtualisation is not isolation, it's sharing
<bslsk05>
en.wikipedia.org: Return-oriented programming - Wikipedia
<epony>
see also the "See also" section
vdamewood has quit [Quit: Life beckons]
[_] is now known as [itchyjunk]
gildasio has quit [Ping timeout: 258 seconds]
gildasio has joined #osdev
<Clockface>
is it considered feasible to relocate flat binaries?
<Clockface>
no one has tried it before
<Clockface>
actually
<Mutabah>
If there's relocation information, then it's not really a "flat" binary
<Clockface>
i meant assuming the whole thing is machine code
<heat>
i mean...
<Clockface>
finding which parts are addresses, and hoping for the best
<Clockface>
just describing it, it seems like a worse idea
<heat>
oh hell nah
<Clockface>
lol
<Clockface>
nevermind then
<Griwes>
just have the flat binary be position independent, easy
<Mutabah>
Write position-independent code, and you're good :)
<Griwes>
great minds think alike
<heat>
decoding the shitshow that is x86 opcodes to find out which instructions reference memory and relocate them? fucking hell
<heat>
you would also not be able to know what is .text and what is .data
<Clockface>
that was my initial idea, PIC
<Clockface>
but then i wondered about being consistent with my PE stuff
<Clockface>
and now i have decided that its probably impossible anyways
<heat>
you can do PIC but your .data still needs to be relocated
<Griwes>
I was about to say, and then finding out that you only decoded 5 instructions right because there was a data byte that threw everything else off lol
<heat>
> being consistent with my PE stuff
<heat>
hm?
<Clockface>
i went with PE
<Clockface>
since i like the DOS/windows world
<Clockface>
also to be more in line with UEFI
<Griwes>
the only way my OS touches PE files is when I objcopy my elf uefi binary into a PE
<heat>
very weird choice
<Griwes>
I'm very happy this way
<Clockface>
there isnt anything wrong with it, and i have that DOS stub
<Clockface>
the DOS stub gives me options
<heat>
what options?
<Clockface>
idk
<Clockface>
but its there
<Griwes>
lol
<heat>
having a DOS stub is a geneva convention sanctioned warcrime
<heat>
i like PE's machine type hexes
<heat>
and that's about it
<Clockface>
i think the file extensions are comforting
<mxshift>
Throw in execute from ROM for even more fun
<bslsk05>
formats.kaitai.io: TE (Terse Executable) file format spec for Kaitai Struct
<heat>
but your bootloader is just your bootloader
<Clockface>
and i already designed it around PE
<Griwes>
the role of a bootloader is to contain all the shitty boot stuff you don't want to live in your pristine kernel
<heat>
what part of your design is designed around PE?
<Griwes>
it's a barrier shielding the kernel and allowing it to completely ignore how it gets booted
<Clockface>
well, not the design
<Clockface>
i just already put work in
<Clockface>
im not changing paths now
<heat>
what did you write already?
<Clockface>
good point
<Clockface>
i have a very detailed road map for what to write in what order and how
<heat>
that's a bad idea
<heat>
you don't know how an operating system works
<heat>
let's put it that way
<Griwes>
if I never changed paths, I'd've be writing essentially a monolithic kernel for the past 11 years
<Clockface>
im basiclaly just making a bootloader for PE files
<Clockface>
and yes i am a noob
<Griwes>
and that would've been *awful*
<heat>
a good chunk of knowing how to draw a roadmap is putting in the work
<Clockface>
ive already made a bunch of random crap
rurtty has quit [Ping timeout: 268 seconds]
<Clockface>
im just not reusing it
<heat>
sketching a masterplan is a poor idea if you don't know what you're doing
<epony>
nowonowooooooo.. you're an apprentice.. you get frenzy.. bleep bloop blumm, goodbye to your world, it's not a moon, it's a butt a big round butt https://www.youtube.com/watch?v=eT4shwU4Yc4
<Griwes>
and going into a thing with a masterplan you are *set* on never changing is like the worst way to plan anything
<Clockface>
alright
<Griwes>
you'll always learn new things while doing new things
<Clockface>
i should just make my fat32 lib first
<Clockface>
by the way
<Clockface>
should i use the bios's way of using the disk or do it myself?
<Clockface>
at first
<mxshift>
Good question. Probably should figure that out before writing filesystems
<Clockface>
well i plan on having an abstract function to handle raw disk i/o
<Clockface>
so i can rewrite that part whenever i want
<heat>
if EFI use efi
<Mutabah>
Just write a basic PIO (or maybe DMA) IDE driver
<heat>
you mentioned uefi
<Mutabah>
First time around, don't over-design - just get to hacking
<heat>
use its simple filesystem protocol
<heat>
EFI skips like all the hassle of writing a bootloader
<Mutabah>
Or yeah, stay in EFI boot services and use its IO
<Mutabah>
(Oh, you're writing a bootloader?)
<heat>
yes they're writing a bootloader
<mxshift>
UEFI at least gets you access to a few filesystem types
<Clockface>
should i make a wrapper around that meant to be my version of filesystem function
<Clockface>
since later on i might want to my own filesystem
<heat>
no
<Clockface>
and it would be annoying to rewrite stuff with the uefi functions in it
<heat>
don't overdesign
<heat>
and it's not like you couldn't write your own filesystem driver if that were the case
<mxshift>
You're going to end up rewriting everything at least once anyway
<heat>
uefi super extensible hehe
<Clockface>
thats what i plan on doing
<mxshift>
I prefer to start small: get some output to a screen or serial port
<heat>
for sure
<Clockface>
ive done that stuff before
<mxshift>
Makes debugging a lot easier
<heat>
that's step1 of porting something to a new arch too
<Clockface>
thats what i mean, ive done crappy experiments to do a lot of things
<mxshift>
Yet you'll need to redo them for your new thing
<heat>
ok, then write the codes
<Clockface>
exactly
<Clockface>
so this time i want to get it right
<Clockface>
so its not another write once and forget
<mxshift>
You won't
<heat>
<Clockface> and yes i am a noob
<mxshift>
There is no such thing as "right" in this space
<Clockface>
its supposed to be good enough to build off of
<heat>
dude
<heat>
shut up and write code
<mxshift>
Even the most horrible bootloader that functions can be built off of
<heat>
this is the best advice i'll be able to give you
<Clockface>
alright fine
<heat>
you're overthinking it
<heat>
overdesigning it
<heat>
and this is *the fucking bootloader*
<heat>
the only thing that is more of a sideshow than a bootloader is firmware itself
<heat>
bootloaders are allowed to be crummy
<epony>
the space is very big.. up to 4TB of RAM and on the rise (it's expanding but deccelerating)
<heat>
Clockface, anyway, set small goals for yourself. load a file in EFI, then figure out how to load the kernel (big tip: use ELF, you have a lot more tooling and the format is IMO a lot simpler)
<heat>
once you did that, get out of efi and into the kernel, then bootstrap yourself out of that, etc
<clever>
if i was doing something like that, i would have my efi application respect the virtual addresses in the elf headers, and generate some paging tables
<clever>
but load them basically anywhere in physical memory
<heat>
that's possible
<clever>
then it can `ExitBootServices`, change the paging tables over, and jump to the kernels _start
<heat>
the Classic(tm) approach is to do it by hand
<heat>
in the kernel's entry
<clever>
setting up the mmu in entirely PIC asm just seems like a nightmare to me
<clever>
just do it in C
<heat>
it's relatively easy
<clever>
and efi already has pre-relocated C code running for you
<heat>
well, "easy"
<clever>
no need to deal with PIC
<Clockface>
one of my specific goals is to use the relocation tables in PE (or i guess ELF)
<Clockface>
i would like to relocate stuff rather than use PIC
<Clockface>
thats the cool part that i really want to do
<clever>
Clockface: one advantage of that route, is ASLR, load the kernel to a random virtual addr on every boot, to harden yourself against exploits
<heat>
Clockface, hm?
<Clockface>
hm?
<heat>
why do you want to relocate rather than use PIC?
<heat>
PIC is highly superior to relocation
<Clockface>
its marginally more expensive once its running
<clever>
heat: from what ive seen in a few random pokes with gcc, it cant really generate 100% PIC code, it always needs some relocation patching
<heat>
right
<clever>
PIC mode just reduces how much patching you need
<heat>
Clockface, relocation is a whole lot more expensive
<Clockface>
only when loading
<heat>
no
<klange>
PIC primarily localizes relocations to tables.
<Clockface>
isnt that the point
<Clockface>
the loader relocates the library or whatever then you can forget about it and pretend it was compiled that way
<heat>
if you would relocate everything (as in code, data, etc), you use up a lot more memory
<heat>
but these are VM details you're not aware of
<clever>
heat: isnt that only true for shared objects?
<heat>
clever, for every object
<heat>
they're all mapped MAP_PRIVATE
<clever>
but the kernel itself, is only ever mapped once
<klange>
PIC is not any slower on modern arches, x86-64 has whole buckets of instructions specifically for supporting the model.
<clever>
so it doesnt pay that overhead
<heat>
clever, sure, but the kernel is different, and Clockface isn't particularly talking about the kernel I think
<clever>
ah, i thought he was talking about kernel
<mxshift>
clever: you almost described PHBL
<Clockface>
i am
<Clockface>
thats why im focused on my bootloader
<Clockface>
its going to be relocating all the kernel bits
<Clockface>
so its kind of important
<clever>
mxshift: what is that?
<mxshift>
Oxide bootloader
<heat>
Clockface, ok sure, that's fine. plenty of kernels just relocate themselves
<clever>
ah
<Clockface>
if anything i think relocation sounds like a better fit for kernels than user libraries
<klange>
The point in user libraries is it is necessary for actually being able to share them.
<Clockface>
since they are not shared, and bootup time is less noticable than a program lagging on startup
<Clockface>
im assuming the kernel doesnt fully know its own addressing since you might load different modules into it
<Clockface>
drivers and all that
<klange>
If you have to write all over your code, you can't share any of it unless it all gets loaded to the same place, which is something that was learned the hard way many years ago.
<Clockface>
yeah
<Clockface>
thats why i think its better for the kernel since it doesnt need to share anything
<heat>
the kernel doesn't necessarily load modules
<heat>
in fact, modules are herd
<Clockface>
wat
<klange>
I did static object modules. It was so fun, I did it three times.
<klange>
heat: don't make me get the spray bottle
<Clockface>
i might do relocation for usermode too, but im specifically interested in it for kernel stuff
<heat>
i also did static object modules
<heat>
they are not fun
<klange>
I like to call it runtime static linking.
<heat>
like, it's OK but I have absolutely 0 need for it
<heat>
and unloading is a whole other can of worms
<klange>
It's cute and 'real' operating systems do it.
<heat>
and relocation... ah fuck that
<heat>
the big no-no is that debugging gets way harder with gdb
<heat>
at least without scripts
<Clockface>
relocation isnt supposed to be an easier way to do things
<Clockface>
i think it probably is slightly harder
<Clockface>
i just think its cooler
<heat>
no
<heat>
it's easy and stupid simple
<heat>
well
<klange>
It's an easier way to do "here's a bunch of compiled objects, you can put them together later and each one can be wherever you want".
<heat>
the objects I and klange are talking about hav eall sorts of fucked up object file relocs
<heat>
so you essentially play linker
<heat>
...at runtime
<Clockface>
yes
<clever>
hence, dynamic linker
<heat>
dynamic linkers have a lot easier relocs unfortunately
<clever>
with `-dr` as shown above, you can inspect the relocation data
<Clockface>
the processor doesnt have to calculate all the addresses at runtime too
<heat>
the processor is fast
<Clockface>
i know its pretty fast now but still
<heat>
the processor does it no problem
<heat>
there's literally almost no performance difference
<Clockface>
what if i want to run it on an older computer
<heat>
it's enabled by default
<klange>
Yes, unfortunately static objects (at least in ELF-land (Rivendell?)) have far more and different relocations than what gets put into shared objects.
<\Test_User>
fast != instant
<heat>
you keep making up scenarios
<clever>
klange: yeah, ive run into that issue
<klange>
Or rather, because it's totally not normal to be doing static linking like this, but very normal to do shared object linking, shared object linking has been designed to be simpler.
<clever>
klange: i think the simplest hack ive seen, is to staticly link your kernel into a .so form
<\Test_User>
if you add up enough minor slowdowns you get much slower code :P
<klange>
You can build a shared object without any of the usual shared object things that normally go into dynamic libraries.
<klange>
You may need to beat your compiler into submission to get it to actually do that, though, I think gcc + binutils both have a bunch of warnings and catches to stop you from doing it because 'in the real world' it just makes broken stuff.
IRChatter has quit [Ping timeout: 268 seconds]
<clever>
that could also be what led to the fixup.dat mess on the rpi
<clever>
instead of using elf relocation data, or a self-relocating binary, the rpi just applies a binary patch to the whole damn elf, headers and all
<clever>
and then loads it to whatever addr the DT_LOAD now points to
<klange>
Hm, I keep missing maxdev but I need to respond about their virtualbox bug... I can't reproduce it on 6.1 with my current builds, but I did find another bug I introduced when I rewrote my signal implementation (because as we all know, signals suck)
<klange>
Why I use a signal for this specific process I do not exactly recall, but when the video driver changes the framebuffer size it sends a signal to an owner process (the compositor, generally) to tell it to reconfigure, and I completely broke this recently in a very C way.
<heat>
that sounds very SIGWINCHy
<klange>
Yeah, it's pretty much a framebuffer SIGWINCH.
<klange>
And it's signal 35... and let's just say, I was not sending signal 35 but rather signal 3...
<heat>
are you self hosting yet?
<Clockface>
not everything is a goddamn linux cloen
<Clockface>
*clone
<heat>
hm?
<klange>
You mean 'again', and I haven't really bothered. I can build _a_ functioning CD with an uncompressed ramdisk, and I can script something to build gcc+binutils and avoid the configure/make mess, so that maybe qualifies...
<klange>
I consider the real missing piece to be actually writing a gzip/DEFLATE compressor, and not a cheating one that just slaps the header on an uncompressed block.
<klange>
(Everything I have that supports gzip-compressed stuff also supports not-gzipped-compressed stuff, so that wouldn't even gain me anything)
<heat>
ah cool, didn't remember you were self hosting at some point
<klange>
I have old videos of building CDs and then running them in bochs :) Now that's a port I need to fix up and ship again...
<heat>
aw jeez not bochs
<heat>
ship qemu
<klange>
but qemu is more work :(
<heat>
more work is good
<klange>
and then someone will be mad I don't have a kvm :\
<heat>
don't forget this is osdev
smeso has quit [Quit: smeso]
heat has quit [Ping timeout: 268 seconds]
smeso has joined #osdev
<klange>
Given that my only hard third-party dep at this point is "a gcc-compatible compiler toolchain", I'm kinda cheating at the idea of being self-hosted, and I haven't actually done the "bitbang a gcc+binutils build from source" thing so that might be a 'worthwhile' endeavour to solidify the claim.
<klange>
But it might also just be easier to get it to build from a pre-configured Makefile now that I have dash...
<klange>
If you want try self-hosting go poke sortie.
<klange>
There's still differences between my self-hosted builds and my normal builds, though most of them are just a matter of laziness at this point, like the aforementioned gzipping of ramdisks - I build them with Python's tarfile module. I also do FAT images for EFI in regular builds, I use xorriso to build CDs instead of spitting out ISO9660 manually (another bit of Kuroko that was previously Python), the whole
<klange>
thing is in a Makefile (instead of glorified shell script that is the self-hosted build)...
<klange>
... oh heat timed out, I guess I'm just rambling.
<klange>
Now that I have that glyph cache in my terminal it's slick and speedy and very usable for local hacking, so I've done some iterative development entirely hosted, then I can do network copies or spit out sources over serial to get them back onto the host and into git. git might be a good port, but I don't think I trust my network stack enough with all the intermediary parts, and I would probably want an ssh
<klange>
client for that anyway and... just buckets of third-party stuff that isn't on my radar at the moment.
<geist>
word.
* kazinsal
. o O ( toarukvm + qemu as an enterprise hypervisor )
bgs has quit [Remote host closed the connection]
[itchyjunk] has joined #osdev
darkstardev13 has quit [Remote host closed the connection]
darkstardev13 has joined #osdev
darkstardev13 has quit [Remote host closed the connection]
darkstardev13 has joined #osdev
darkstardev13 has quit [Remote host closed the connection]
darkstardev13 has joined #osdev
darkstardev13 has quit [Remote host closed the connection]
darkstardev13 has joined #osdev
darkstardev13 has quit [Remote host closed the connection]
darkstardev13 has joined #osdev
darkstardev13 has quit [Remote host closed the connection]
darkstardev13 has joined #osdev
darkstardev13 has quit [Remote host closed the connection]
darkstardev13 has joined #osdev
darkstardev13 has quit [Remote host closed the connection]
darkstardev13 has joined #osdev
darkstardev13 has quit [Max SendQ exceeded]
darkstardev13 has joined #osdev
darkstardev13 has quit [Max SendQ exceeded]
darkstardevx has joined #osdev
darkstardevx has quit [Remote host closed the connection]
[itchyjunk] has quit [Read error: Connection reset by peer]
darkstardevx has joined #osdev
IRChatter has joined #osdev
heat has joined #osdev
<heat>
oh shoot sorry klange for leaving you
<heat>
yeah I'm almost self hosting
<heat>
I can reliably build my kernel
<heat>
I'm just missing gn (ported, not tested) and ninja (not ported yet), plus some utilities maybe
<heat>
ofc I really want this to be *reliable*
<heat>
and fast, but that's another metric
<heat>
it spends most of a make vmonyx -j8 blocked which... yeah kinda sucks
<heat>
I get peak CPU usage of 200%
<heat>
i also haven't tested any of this on a real filesystem which... yeah
<heat>
filesystems are herd
<mjg>
where is it blocked
<heat>
stupid mutexes and IO as far as I can see
<heat>
things get quite a bit better once gcc gets mostly faulted in but it's still a locking nightmare in my vm objects
<heat>
also my address spaces are mutexed, so no concurrent faulting
<heat>
i'll need to rwlock those
<heat>
not that gcc is particularly threaded outside of LTO
<heat>
I could also use a proper clone(2) or rfork(2)
<heat>
chocking news: it turns out vfork'ing is a hell of a lot cheaper than fork and you should totally use it
<heat>
shocking*
<mjg>
who knew
<Mutabah>
spawn or vfork :)
<heat>
yeah posix_spawn usually uses vfork-like mechanisms
<heat>
but notice the *-like*, because it can't safely use vfork itself, it needs to switch stacks somehow
<heat>
musl uses clone(2)
<heat>
it uses the same semantics as vfork but switches stacks in assembly
<heat>
i could also implement posix_spawn in the kernel but screw that
<mjg>
netbsd moment
<heat>
sometimes i ponder supporting lwp-like threads
<heat>
which for some reason everyone in relevant-ish UNIX does
<mjg>
it's not what you think
<mjg>
it's a lolo hack which did get cleaned up
<mjg>
and it comes with idiotic problems in linux
<mjg>
did not*
<klange>
I had a fun performance problem a while back where my symbol resolution in my ld.so was set up for ~hundreds of symbols and gcc was coming in with thousands and it took forever to load
kindofwonderful has joined #osdev
<kindofwonderful>
it's me folks i have identity crysis
<kindofwonderful>
some guy at some channel don't like me
<kindofwonderful>
unfortunately he also have channel priviliges
<kindofwonderful>
what a surprise huh ?
GeDaMo has joined #osdev
heat_ has joined #osdev
heat has quit [Read error: Connection reset by peer]
heat has joined #osdev
heat_ has quit [Read error: Connection reset by peer]
CryptoDavid has joined #osdev
<kindofwonderful>
going to work out then i try to advance my C skills
<kindofwonderful>
my ultimate goal is to #osdev
<kindofwonderful>
Cya
<sham1>
What
elastic_dog has quit [Killed (molybdenum.libera.chat (Nickname regained by services))]
elastic_dog has joined #osdev
netbsduser has joined #osdev
vdamewood has joined #osdev
funk101 has joined #osdev
funk101 has quit [Quit: Leaving]
CryptoDavid has quit [Quit: Connection closed for inactivity]
nyah has joined #osdev
hellstabber5 has joined #osdev
pg12_ has quit [Ping timeout: 252 seconds]
pg12_ has joined #osdev
heat has quit [Remote host closed the connection]
heat has joined #osdev
rurtty has joined #osdev
maxdev has joined #osdev
Terlisimo has quit [Quit: Connection reset by beer]
Terlisimo has joined #osdev
maxdev has quit [Quit: Lost terminal]
maxdev has joined #osdev
CryptoDavid has joined #osdev
dude12312414 has joined #osdev
dude12312414 has quit [Quit: THE RAM IS TOO DAMN HIGH]
theruran has quit [Quit: Connection closed for inactivity]
jjuran has quit [Quit: Killing Colloquy first, before it kills me…]
jjuran has joined #osdev
kindofwonderful has quit [Quit: WeeChat 3.6]
heat has quit [Read error: Connection reset by peer]
heat has joined #osdev
maxdev has quit [Ping timeout: 268 seconds]
elastic_dog is now known as Guest1461
Guest1461 has quit [Killed (tungsten.libera.chat (Nickname regained by services))]
elastic_dog has joined #osdev
xenos1984 has quit [Ping timeout: 268 seconds]
xenos1984 has joined #osdev
maxdev has joined #osdev
<maxdev>
okay so I'm trying to get my toolchain ready for x86_64, I'm building gcc with --enable-shared, got it configured so it uses shared libgcc and everything builds fine *except* for libstdc++-v3; there I get "checking for shl_load... configure: error: Link tests are not allowed after GCC_NO_EXECUTABLES"
<maxdev>
I guess it's because I don't have dlopen because my kernel does the dynamic linking at the moment, but how can I tell configure that it shouldn't use it?
<maxdev>
for i686 it all works as expected though
CryptoDavid has quit [Quit: Connection closed for inactivity]
<maxdev>
klange sorry to bother you directly, but judging from your commit message :P did you maybe have a related issue with libstdc++?
<maxdev>
aha. got it. the issue was that my x86_64-ghost-gcc failed to build executables due to a missing symbol in my libc which only happened on x86_64
<zid>
nod GCC_NO_EXECUTABLES sounds like you failed the "can the compiler make things?" step
<zid>
which is the only way i've ever had ./configure fail ever :P
wootehfoot has joined #osdev
heat has quit [Remote host closed the connection]
heat has joined #osdev
<zid>
oh the insane guy is back
<zid>
asking the same nonsense question
<zid>
How does kernel decide if use cpu or gpu after fetch decode excecute phase?
<mrvn>
why would the kernel decide anything there?
<LittleFox>
or even how, but I guess that's the point - someone asked that but that just makes no sense at all
<mrvn>
that too. Do you think there is a separate fetch decode execute unit?
<mrvn>
what would the cpu or gpu even do AFTER execute?
<mrvn>
zid: who is this insane guy you are talking about?
<j`ey>
heat: i opened a file outside of arch/arm64 today :o
<maxdev>
zid: yep that's exactly what it was
genpaku has quit [Remote host closed the connection]
genpaku has joined #osdev
dude12312414 has joined #osdev
<heat>
j`ey, oh wow
<heat>
how do you feel?
<j`ey>
heat: scared!
<j`ey>
something going wrong with signals I think
wootehfoot has quit [Quit: Leaving]
<heat>
oh wow that is scary
<heat>
what's going wrong?
<j`ey>
well first it seemed like something was trying to execute from address 0, and then I tried another binary and that tried to execute from a PC that wasnt aligned
<heat>
why would all these binaries get signals?
<j`ey>
SIGCHLD, it was busybox ash, after I tried 'ls' or any other program
<heat>
hrm
<heat>
arm64 right?
<j`ey>
yeag
<j`ey>
*yeah
<mrvn>
how can the PC be unaligned?
<heat>
is it possible you hardware dun the fucked up?
<heat>
does this happen with -stable?
<j`ey>
oh no, this is with my changes on top :P
<heat>
can you post the patches
<heat>
or is it TOP SECRET
<j`ey>
no
<heat>
TOP SECRET then
<heat>
cringe
wootehfoot has joined #osdev
<j`ey>
it is what it is
<heat>
never liked arm anyway
<heat>
more of a leg guy myself
<zid>
heat enver skips leg day
<zid>
just because he hates arm day
<j`ey>
mrvn: it's just one of the exceptions?
<mjg>
#skiplegday is a t-shirt i would buy
<heat>
nah leg day is fun
<zid>
I can only do leg day, my arms are twigs
<maxdev>
i recently skipped them less frequently
<heat>
mjg, btw i wasn't implying lwps are a good idea
<heat>
it's just A Thing you can support without much difficulty
<heat>
the concept of processes that share a bunch of shit isn't controversial
<heat>
if you take it to the extreme, you have a weird process thread
<heat>
if you add some hacks around it, tada, you have NPTL
<mjg>
i'm sayin everyone had this shit initially
<mjg>
but not everyone cleaned it up later
<heat>
none of the main players seem to have cleaned it up
<heat>
unix at least
<mjg>
SOLARIS did
<heat>
praise be
<heat>
something that really annoys me is ptrace semantics and threads
<mjg>
to give them some credit, they did clean up some of the original unix nastyness
<heat>
it works /wellish/ with thread processes
<mjg>
i'm not touching ptrace
<heat>
ptrace is very nasty
<heat>
the interfaces sucks ass
<heat>
it's named ptrace everywhere but the semantics vary wildly sometimes
<heat>
it's also a syscall that multiplexes into multiple other functions
<heat>
the arguments get overloaded
<heat>
it's depressing
<mjg>
see, while i acknowledge ptrace is crap, i have to say i don't know how to make one from scratch which would be good
carbonfiber has quit [Quit: Connection closed for inactivity]
<heat>
more than a syscall would be nice
<heat>
separating pid and tid too
<heat>
hell, even a ptracefd would be better
<zid>
I've never looked at it
<zid>
my default reaction would be that you get a blocking fd that you read from, having subscribed to various events
<zid>
and the events fill the fd with proto bytes like gdb
<heat>
ptrace does more than that, it's way more "interactive"
<heat>
setting breakpoints, trace start+stop, etc
<zid>
not sure if I'd make the write side async, or make you need to ack + response
<zid>
maybe both!
<heat>
i like how well designed unix is
<heat>
r/newsentences
<heat>
if everything is a file why isn't tracing a file descriptor and why can't you open it in the filesystem
<heat>
"everything is a file but we give up halfway there"
<zid>
If everything is a file, why can't I delete the recycle bin
<heat>
idk
<zid>
checkmate theists
<heat>
you can probably blame that on VMS or whatever
<mrvn>
zid: On AmigaOS you could.
<heat>
dave cutler hated unix so much he made the recycle bin undeleteable
<sham1>
Can you blame cutler
<sham1>
Probably not
<heat>
<heat> never liked arm anyway
<heat>
<heat> more of a leg guy myself
* heat
applies to a job at ARM
<heat>
I've always been an arm person
<heat>
ever since I was 3 years old, in 1976
<heat>
x86 bad riscv bad loongarch bad arm good arm64 best
<heat>
cambridge is the best city in england
<netbsduser>
the simple trick to appreciating Unix is to recognise actually-existing Unix as not the best implementation of ideal Unix
<sham1>
Real Unix has not been made yet
* sham1
runs
<netbsduser>
"everything is a file [descriptor]" is a fantastic concept and has been used to great ends, e.g. FreeBSD's capsicum which provides an easy path to introducing the object-capability security model to unix on the basis of file descriptors
<heat>
we have never achieved true unix
<heat>
just like communism
xvmt has joined #osdev
<netbsduser>
heat: good work on noticing my allusion to East Germany's apologetic self-description as "actually existing socialism"
<netbsduser>
this was their retort against those who criticised them from the left
<netbsduser>
in the case of unix the situation is inverted. the 'unix way' was a post hoc rationalisation of its design. meanwhile there were attempts at being a Unix more Unix than Unix. Mach's 'port' concept was such an attempt to really generalise the file descriptor into a universal abstraction on which everything is based
gog has joined #osdev
<sham1>
I do think that in some ways Plan9 is better. Of course Plan9 was explicitly made in Bell Labs based on the lessons from UNIX but still
<dzwdz>
plan9 wasn't real unix either
<netbsduser>
plan9 has some nice aspects
\Test_User has quit [Quit: \Test_User]
<sham1>
Well Plan9 at least went all in with the "everything is file descriptor" and just made everything into file trees
<sham1>
A microkernel in the style of P9 would certainly be something. Of course microkernels (at least orthodox ones) I feel have some major bootstrapping problems
<netbsduser>
kell also gave a related talk on it called "Liberating the Smalltalk lurking in C and Unix"
<dzwdz>
so instead of doing `mount /dev/sata/0 /root` or something like that you did `mount #s /root`
<sham1>
Oh
<dzwdz>
you can't really replace that #s with custom stuff as you can with stuff on the filesystem
<sham1>
Oh that's a shame
<dzwdz>
which is horrible - if they did that then plan9 would have gotten sandboxing perfectly right like 20 years ago
<netbsduser>
dzwdz: a terrible missed path
<dzwdz>
exactly the one i'm exploring in my kernel
<dzwdz>
:p
<netbsduser>
we would have flying cars and colonies on Mars if only we had implemented capabilities-based approach to sandboxing historically, instead of going down the false path of byzantine access-control lists which have given us little of any value to date for the average desktop user
<dzwdz>
exactly
<dzwdz>
i feel like i've gotten sandboxing about right in my design
<dzwdz>
you can do `whitelist /net/listen/0.0.0.0/tcp/80 /bin/httpd:ro /var/www/:ro -- httpd` and it works about as you'd expect
<dzwdz>
the entire whitelist command being like 100 lines of code
heat has quit [Read error: Connection reset by peer]
heat_ has joined #osdev
<sham1>
The irritating part about hobby OSDev is that it's like the worst thing for a perfectionist
xenos1984 has quit [Ping timeout: 246 seconds]
maxdev has quit [Ping timeout: 252 seconds]
<gog>
hahahaha yeah
zxrom has joined #osdev
<gog>
.... yeah
* gog
stares at her awful code disdainfully
<sham1>
You at least *have* code. Meanwhile I have had so many "false starts" with this stuff that I'm starting to lose faith in being able to do stuff
<sham1>
Of course for me, that's not *just* a thing with OSDev but like all of my hobby projects
<sham1>
I just get distracted and then it's all downhill from there. And even if I've written some code before, once I get back to it I realise how terrible it is and want to start over
<mjg>
ze journey is ze destination
<gog>
bold of you to assume my existing code isn't yet another false start of my own
* Ermine
cheers up gog
<gog>
:P
<gog>
thanks
xenos1984 has joined #osdev
wootehfoot has quit [Read error: Connection reset by peer]
<zid>
imagine finishing anything
heat has joined #osdev
heat_ has quit [Read error: Connection reset by peer]
<sham1>
To get a bit more off-topic for a second, I think that this might be some sort of unconscious fear of failure or some such. Especially since with things like this, you're only really accountable to yourself. No deadlines, no external motivation necessarily, nothing. There's no cost to failure with this, yet it's still something that haunts ya
<sham1>
With me and for my OSDevving it usually manifests in things that realistically are rather trivial, like build systems and even like the most unspecific kinds of architectural decisions
<sham1>
(Wrt. buildsystems it's "trivial" in the sense that one could just use something like Makefiles and be done with it)
\Test_User has joined #osdev
<dzwdz>
operating systems are way too complex for a single person to get everything perfect
<sham1>
Well yes, but the thing is that perfectionism is hardly a rational thing
<dzwdz>
you gotta embrace the chaos
<zid>
You can perfectionism, just don't expect to get much done
<zid>
some people spend 20 years writing a bootloader :P
<zid>
some people copy paste from stackoverflow and get 3D spinning cubes in 3 days
<dzwdz>
you could just focus on making everything as easy to replace and work on further as possible
<dzwdz>
cover enough ground quickly to get your os useable, and then start working on making it actually good
wand has quit [Quit: leaving]
\Test_User has quit [Quit: \Test_User]
Guest1982 has joined #osdev
Guest1982 has quit [Client Quit]
Test_User has joined #osdev
<heat>
sham1, just do it
<heat>
like nike
<heat>
and shia labeuof
<sham1>
I immediately thought of the latter
<heat>
wanna come to onyx?
<heat>
unpaid internship
<heat>
you'll learn from the best a kinda ok os dev
<heat>
i'll pay you in exposure and motivation yes?
<sham1>
pfft, I've heard this one before
<sham1>
But yeah, I probably should just do it
<sham1>
Actually start
<sham1>
I do have some ideas after all
Test_User is now known as \Test_User
vdamewood has quit [Quit: Life beckons]
<heat>
yeahhhhhhhhhhh
<heat>
go for it big man
<heat>
or woman, or they
<geist>
just do it (TM)
<heat>
planning is bad
<heat>
futile really
<heat>
nothing can prepare you for the chaos of real life
IRChatter has quit [Ping timeout: 268 seconds]
<geist>
really i think most of you have missed the golden era of osdev, honestly. there was something much simpler about hacking on basically 386/486 era hardware, and it being mostly a black box
<geist>
it was a pain, you had to write a floppy, and you realistically needed a second computer
<heat>
wdym black box?
<geist>
but there'sstill something really pleasing and nice about that level of simplicity
<geist>
well okay not a black box, but there wasn't a huge wealth of information easy to get
<heat>
oh yeah for sure
<geist>
you had the intel manual if you could get one or get it printed out
<heat>
it's super impressive how torvalds just wrote a unix
<geist>
and you had like the ralph brown interrupt list
<heat>
from scratch
<heat>
in the fucking 90s
<heat>
the code was shit but It Just Worked(TM)
<geist>
yeah but my point is the hardware was sooo much simpler
<heat>
you know, yes and no
<geist>
there were less assumptions about detecting stuff
<heat>
I kind of disagree with that statement
<geist>
it was more okay to just hard code that a floppy and an ide controller was there
<heat>
I find it much more difficult to drive old ass hardware than new
<geist>
so hobby os level 'just start talking to the keyboard and floppy as if it were there' was just assumed
<geist>
it's fun now, but so much stuff is dynamic nowadays. it's more standardized (ahci, xhci, etc) but it's also much more complicated and dynamic
<geist>
so it's a tradeoff
<heat>
yeah
<heat>
give me my static device files :P
<geist>
it's kinda a golden era now, but mostly because we have all the tools available. but to a certain extent the sky is the limit
<geist>
and that's kinda a source of anxiety, kinda. like anything is possible, it's just you have to do it
<sham1>
That is indeed a lot of it
<geist>
but back then the hw limitations were more real so you had a narrower scope
<heat>
i think writing something that realistically competes with existing "professional" stuff these days is stupid hard
<heat>
as you've seen with fuchsia
<geist>
yeah, exactly
<netbsduser>
for some reason i went so far as to get info on i/o port and irq assignments for the ps/2 controller (if it exists) from acpi
<geist>
or just in general with you and mjg going on for days about optimizing this or that
<heat>
torvalds competed with UNIX in like, 2 years
<heat>
yeah totally
<heat>
not even only talking about performance but features, security, etc
<heat>
all the goodies you come to expect from an OS
<geist>
right
<heat>
30 years of continuous development is a *lot*
<heat>
early 90s was really the golden era if you wanted your thing to be used IMO
<netbsduser>
linux's success ought to inspire anyone
<netbsduser>
they had to call in Matt Dillon to advise them on why their vmm was as slow as mud, and he suggested that "scanning every address space for mappings to a page whenever you want to update its mappings for CoW or swapout" was not a good idea
<netbsduser>
this they finally dealt with a full decade after Linux's first release, and after linux was already prolific
<heat>
yeah
<heat>
you understand what I mean then?
<heat>
it was easy(ish) to get people to switch to your hobbyish thing
<heat>
now these days no one is going to rally behind a rando operating system
<netbsduser>
yeah, you are completely right, historical circumstance has proven to be the main thing that dictates whether you'll win
<heat>
even the BSDs have a severe lack of manpower and attention
<netbsduser>
i think in the late-00s was when any chance of a small player ever winning big was extinguished
<j`ey>
heat: how many other hobby ish OS's did that happen with?
<netbsduser>
i remember some random proprietary unixlike called skyos somehow got a quite large following around then, and ported dozens of major softwares, including Firefox
<heat>
386BSD was developed by like 2 people after forking from 4.4BSD
<heat>
how many people did CSRG have? like 10-20?
<heat>
also minix, linux
<netbsduser>
while the most impressive and wide-reaching hobby OS i know of today, managarm, has had a big struggle porting chromium
<j`ey>
is managarm bigger than serenity?
<heat>
probably
<netbsduser>
in scope and ambition, yes
<heat>
at least in quality, maybe not contributors
<heat>
chromium requires like, almost everything
<netbsduser>
the focus is almost the opposite of serenity
<heat>
it's a bit stupid
<heat>
opengl implementation, windowing, sound, input, and a solid bit of everything
<netbsduser>
heat: even to the likes of openbsd and netbsd it's a big struggle to maintain
<heat>
and I'm probably missing some requirements
<klys>
usb, video
<netbsduser>
google refuse patch submissions for some inane reason
<heat>
maybe they feel like no one uses net and open
<heat>
which is /partially/ true
<klys>
someone should repo those submissions on a site then
<netbsduser>
i think some freebsd submissions were even rejected, but the limbs are working against each other there
<netbsduser>
after all, google funded and supplied engineers to the work of making Chromium work with FreeBSD's capsicum framework
<heat>
big corp moment
<heat>
intel develops like 3 firmware products that compete with each other
maxdev has joined #osdev
<heat>
>for some reason i went so far as to get info on i/o port and irq assignments for the ps/2 controller (if it exists) from acpi
<heat>
+1 for doing it right
<heat>
x86 does not have platform devices
<heat>
(in theory)
<heat>
new intel platforms don't even emulate PS/2
<klys>
antiproject.cc is unregistered, and theysaidno.cc, and toughluck.cc, ...
<heat>
i don't see why you're trying to make the patch refusals a big deal
<heat>
they don't want to maintain code for platforms that are barely used
<heat>
particularly for general chromium stuff
<heat>
and that's fine
<sham1>
Speaking of firmware, since Intel is removing the legacy PIT if they haven't already done so, I wonder how OSes would synchronise their clocks and to know how many ticks to wait in order to get an interrupt, say, a microsecond or a millisecond from now
<heat>
>Intel is removing the legacy PIT
<heat>
hm? source?
<heat>
aside from that, hpet still has a fixed frequency
<sham1>
Maybe I'm misremembering but I do recall something about that. I don't have a source
<sham1>
Ah
<sham1>
Hpet. Hm
<heat>
new CPUs give you the tsc frequency through cpuid 15h
<heat>
(newish)
<heat>
also on real hardware you can kind of calculate the local apic timer frequency from an msr(right? or is it another cpuid leaf?)
<sham1>
Right, so there are ways to get the frequency. Good to know
<heat>
hpet should always be your first option when calibrating the timer btw
<sham1>
Yeah probably. I'm just curious
<heat>
and i highly doubt they're removing the PIT
<heat>
at least I've never heard of that
<heat>
and it makes very little sense IMO
<heat>
the PIT is stupid simple, it's not wasting much sillicon space
<sham1>
Well I know they're certainly thinking about removing some "legacy devices" given that they've already deprecated the legacy BIOS
<heat>
that has nothing to do with it
<heat>
the legacy BIOS has been deprecated for 12 years
<CompanionCube>
iirc intel has shipped stuff without the PIC, so probably same without the PIT.
<sham1>
Right. Yeah, that's probably what I was thinking of
<klys>
i/o below 0x400 will just have to fit acpi's model of citizenship
<heat>
maybe "stuff", but general platforms? nah
<heat>
ps/2 was "removed" as in they removed the shitty SMM code behind it
<sham1>
And of course with Intel's silicon, given the whole 12nm++++++++ deal, it might be significant. Although that's probably just bants at this point
<heat>
intel platforms are always remarkably similar
<heat>
they keep their io ranges and registers kind of constant
<heat>
shutting down using 0xcf9 works in 2008 and it works in 2022
<bslsk05>
www.mail-archive.com: [PATCH] clocksource: Untrust the clocksource watchdog when its interval is too small
theruran has joined #osdev
h4zel has quit [Ping timeout: 260 seconds]
Piraty has quit [Quit: -]
Piraty has joined #osdev
dutch has quit [Quit: WeeChat 3.6]
wand has joined #osdev
nlocalhost has quit [Remote host closed the connection]
Gooberpatrol66 has quit [Remote host closed the connection]
heat has quit [Read error: Connection reset by peer]
heat_ has joined #osdev
nyah has quit [Ping timeout: 246 seconds]
<klange>
maxdev: it's been about a year since I last touched libstdc++ and it was to remove it from my default build (I build it separately as a package for my repository); all of my work for building it in the first place is from 2018, so I don't really remember what went into it
<klange>
maxdev: btw I was unable to reproduce your issues with clicking in VirtualBox, I tried 6.1.38 and the newest 7.0 that just came out with both my last release ISO (2.0.1) and current builds, on both Windows and Linux hosts and clicking works fine (but I did find a different issue with display resizing that appeared recently, and fixed that...)
maxdev has quit [Quit: Lost terminal]
<klange>
oh no i killed him :O
<klange>
1:30am in Germany, I suspect my messages were causing pings and a sleepy maxdev rolled over in bed and closed the offending terminal.
gog has quit [Ping timeout: 268 seconds]
SpikeHeron has joined #osdev
heat_ is now known as heat
<heat>
oeprating systm
netbsduser has quit [Remote host closed the connection]