foudfou has quit [Remote host closed the connection]
catphish has joined #osdev
<catphish>
Just coming to play with osdev after not doing so for a good few years and i have a super newbie question - does it make the most sense just to begin an OS as an EFI binary now? is there any good reason why you'd start in a more traditional manner, or use an intermediate bootloader?
<clever>
catphish: a lot of people tell you to not start with a bootloader
<clever>
personally, i would just use grub or efi to get my kernel loaded
<clever>
if i was on such a platform
<zid>
absolutely avoid writing a bootloader, whether you want to do anything meaningful with uefi is up to you
<zid>
it's a bit annoying to set up compared to just letting grub multiboot you, if you just wanna dive in and get a 'something' running
<catphish>
clever: i'm not at all interested in a bootloader, what i wondered is whether it was sensible to start by making an EFI binary
<zid>
(and writing a bunch of uefi code just to have 'nothing' run successfully is a bit of a letdown)
<catphish>
or is it more sensible to use an existing bootloader
<zid>
efi is the bootloader there, but you need to set up a PE cmpatible toolchain in order to use it
<catphish>
i looked briefly at EFI, but it seems you need to start with a pretty bulky build with standard libraries
<zid>
It's just headers
<catphish>
ok
<clever>
yeah, EFI doesnt really have dynamic linking, so you need a static .exe file
<zid>
the efi is the standard library, you use a little loader that fills it out via function pointering, all windowsy COM style nonsense ofc
<clever>
so if you use strcpy for example, then you need to link something that implements it, such as a libc.a
<zid>
cus it's.. microsoft's efi
<catphish>
i will try to find out more about what's in such a file
Turn_Left has joined #osdev
<clever>
basically, the efi firmware in the board will load your .exe to a free block of ram, and apply standard .exe relocations to it for that addr, and then jump to your entrypoint
<clever>
and pass you 2 pointers, one to a table of function ptrs
Turn_Left has quit [Read error: Connection reset by peer]
<clever>
and then you can basically do whatever you need to prepare for taking over the system, and when your ready, call the ExitBootServices() function pointer
<catphish>
that seems reasonable
<zid>
efi headers will be #define ExitBootServices(x) (ptr)->vtbl[34](x) nonsense or whatever
<catphish>
it sounds like (based on my own tastes) that i just want to find the lightest possible recipe to build such an executable
<clever>
one potential trap, is the ABI for the efi functions
<zid>
I always fuck that up when I use dlls on windows
<clever>
there is a thing you can tag all function pointers with, so gcc uses the right calling convention
<zid>
DLLSPEC vs WINAPI
<clever>
the other option is to just use a gcc that defaults to that ABI
<clever>
i kind of like the tagging method
<clever>
__attribute__((ms_abi))
<clever>
thats the one
<catphish>
is GNU-EFI the headers i want?
<zid>
proper windows headers just use the defines frome arlier
<catphish>
or can i go even lighter?
<zid>
DLLSPEC and WINAPI and it resolves to __attribute((stdcall)) or whatever is needed
<zid>
you could write your own as and when you need each function/type
<zid>
that's what I do for opengl >_<
<clever>
catphish: i forget exactly what gnu-efi contains
<catphish>
right now i just want the lightest way to create a binary it will acually load, and go from there
<heat>
>because you are fucking with filesystems which might have a way to optimize it
<heat>
not ext4, don't care, seen
<zid>
hea
<zid>
this is a very strange choice of words for talking about day 2
<mjg>
> I also use a lot of CPU, allocate as much RAM as I can get away with,
<mjg>
lmao
<mjg>
literal webdev
<mjg>
if this is in any way faster than copy_file_range, it is a bug in said syscall
<heat>
what
<heat>
are you stupiden
<heat>
you're saying using a lot of CPU and allocating a lot of RAM is webdev, but then you advocate for fancy algorithms so it scales to 50000 cpus
<zid>
webdev is where you use microthreads that do almost no work, but have high overhead
<zid>
then start 40 million of them
<heat>
fwiw nodejs doesn't use threads but an event loop
<Ermine>
I'm copying stuff right now. And throughput on a large number of smol files is just cringe
<mjg>
heat: you missed the part where optimizing for smp often *reduces* both memory and cpu usage
edr has joined #osdev
<mjg>
trivial example: you add rcu, a bunch of locks disappear from the fast path => cheaper to execute in all cases
<mjg>
if you have the opposite experience it's only cause you suck
<heat>
rcu is a funny examplen but at the end of the day it has funny tradeoffs
<mjg>
it was a singular, what's up with n suffix
<heat>
you need to batch or it's super expensive, you can't batch too many or the latency shoots up and it's super expensive for latency-bound shite, if you don't batch enough you'll have lots of memory stuck waiting for the callbacks to execute
<heat>
callbacken
<zid>
this is a very strange choice of words for talking about day 2
<heat>
the ez example for my point is the maple tree
<mjg>
of course there are tradeoffs
<mjg>
on the other hand there are real workloads where it's a clear cut win
<heat>
which scales a lot better than rb trees, but has a significant performance penalty in single threaded workloads
<heat>
due to rcu exactly
<mjg>
so for example PATH LOOKUP is way cheaper with rcu
<mjg>
innit
<heat>
sure
<mjg>
i doubt extra overhead from mapple syroup is inherent, but it is lpausibly true
<heat>
i'm not saying RCU is bad, at all
<mjg>
even then is not the point of mpale syroup that most ops are in fact /cheaper/ single threaded?
<heat>
but saying io_uring cp is WEBDEV is like saying RCU is webdev
<mjg>
i am not saying io_uring is webdev
<mjg>
14:15 < mjg> > I also use a lot of CPU, allocate as much RAM as I can get away with,
<mjg>
i said this is webdev
<heat>
mjg, for the btree sure, i think
<Ermine>
Ah, so there's no point of porting rcu to minix due to lack of smp
<heat>
but as far as I understand maple trees need to constantly kmem_cache_alloc new nodes due to RCU, and it's there where you pay the cost
<heat>
hence the hackery they want to introduce in SLUB
<heat>
but that's not webdev if your entire workload involves moving files fast from one end to another
<mjg>
not webdev is to use copy_file_range
<mjg>
then you are not cpu intensive and ram usage is not much oa thing to worry about either
<mjg>
it's literally a self-induced problem to copy "by hand" after the syscall was introduced
<heat>
i mean, last I checked GNU cp uses copy_file_range when it can
<mjg>
14:16 < mjg> if this is in any way faster than copy_file_range, it is a bug in said syscall
<Ermine>
1) how different is it from splice?
<mjg>
this is so webdev i'm surprised it's not in RUST
<Ermine>
2) You still need to open() both ends, so it will choke on "lots of smol files" case
<Ermine>
Idk if io_uring would help though
Turn_Left has quit [Remote host closed the connection]
<netbsduser>
the I/O U-Ring
<heat>
io uring can do that shit ASYNC
<heat>
pretty sure it can walk paths async too
Turn_Left has joined #osdev
<Ermine>
nice
<heat>
io_uring is nice and fast and async-friendly
<heat>
i don't know what causes it to be faster than copy_file_range
<heat>
but it seems to be, for some reason, at least in this guy's tests
<zid>
this is a very strange choice of words for talking about day 2
<mjg>
heat: so i created a 2G file and this webdev program ate 2G RSS to copy it
<mjg>
utterly idiotic is what it is
<mjg>
in contrast cp had rss at 2MB
<heat>
cool stuff
<heat>
now flamegraph
<mjg>
comparing real time is a problem though, it wildly differs between runs
<mjg>
presumably there is off cpu time which fucks it up
<Ermine>
lol protobuf company
<mjg>
however, shortest time was on cp with 1.5s
<mjg>
and 2s for wcp
<mjg>
worst time is > 5s both
<Ermine>
megalul
<zid>
google cpus run protobufs as bytecode
<heat>
mjg, that's good to know
<mjg>
slurping entire file upfront is idiomatic rust
<mjg>
again surprised it is c++ instead
<heat>
i don't know what the guy was doing that copy_file_range sucked
<heat>
what fs?
* Ermine
slurps stuff in go
<mjg>
ext4
<heat>
is src and dest in the same filesystem?
<mjg>
note my test is one file copy
<mjg>
big one too
<mjg>
meanwhile he was running for tons of small files
<mjg>
i suspect it was mostly faster dir traversal in wcp case
<mjg>
which is not a factor in my test
<mjg>
bottom line tho, he could have both fast traversal *and* copy_file_range
<heat>
can you copy_file_range from io_uring? probably?
<mjg>
no idea, i suspect if you can't now, you would be able to down the road
<Ermine>
btw, why can't io uring take fs-specific stuff into account?
<mjg>
why would you even try to do that
<mjg>
seems like layering violation
<heat>
what?
<heat>
what do you mean?
<mjg>
i was responding to Ermine
<heat>
yes, what does Ermine mean
<heat>
btw the protobuf company does not have io_uring enabled in their kernels
<heat>
so wcp is unfortunately not usable there :(
<heat>
"In June 2023, Google's security team reported that 60% of Linux kernel exploits submitted to their bug bounty program in 2022 were exploits of io_uring vulnerabilities"
<Ermine>
Like, btrfs has CoW, idk how to handle it
<bslsk05>
lore.kernel.org: [PATCH 31/31] sched_ext: Add a rust userspace hybrid example scheduler - Tejun Heo
<heat>
real life is a rust-ebpf circlejerk
<netbsduser>
linux must become a microkernel composed of a runtime for EBPF and a suite of at least 9001 eBPF modules to provide all functionality; all to be written in Rust
<Ermine>
hybrid kernal then
<mjg>
> This scheduler demonstrates dividing scheduling logic between BPF and
<mjg>
userspace and using rust to build the userspace part.
<mjg>
wtf
<mjg>
joke which got out of proportion
<heat>
the guy got NAKed in july and is still posting new versions of the patch set
<Ermine>
how many NAKs?
<zid>
I'm a nevernude, so my patches always get accepted
roper has joined #osdev
xenos1984 has joined #osdev
bauen1 has quit [Ping timeout: 256 seconds]
<mjg>
Ermine: one, from peterz!
<mjg>
although the rust + userspace part is new i think?
<mjg>
which only strenghtens the nak tho
<heat>
no, that patch is from the OG RFC
<heat>
i don't know if they kept it on the newer versions
<zid>
heat
<zid>
day 2
<zid>
if you got stuck, at least talk about it
<zid>
instead of going radio silent and ignoring me 4 times
goliath has quit [Quit: SIGSEGV]
<heat>
i'll do it
<heat>
eventually
<heat>
fuck it i'll do it now
<heat>
ok part1 is dun
<zid>
cool
<heat>
part 2 looks... okay?
<zid>
yea
<zid>
I think they changed how it was going to work and made it simpler if I had to guess
<zid>
the ; are suspicious
<heat>
i don't get how type inference works in this language
<heat>
let a = 0; sometimes works but not always
<heat>
yeah day2 was easy
<heat>
just had to come to terms that string parsing was going to suck
<Mutabah>
Also, for `id` you could use `for (idx,line) in input.lines().enumerate()`
<zid>
Day 2 is the earliest I have ever had to use strtok
<zid>
what is this, python!?
<Mutabah>
only the good parts :)
<heat>
you haven't seen the .zip.map parts yet, zid
<heat>
rust has no idea if it wants to be functional or procedural or object oriented
<zid>
No, it also has some of the worst parts
<heat>
it's like C++, feels right at home
Left_Turn has joined #osdev
Turn_Left has quit [Ping timeout: 245 seconds]
<zid>
heat: day3 is the fun one, shall we pencil it in for 6pm?
zetef has joined #osdev
dude12312414 has joined #osdev
<Ermine>
c++ but MeMoRy SaFeTy
<Ermine>
and FeArLeSs CoNcUrReNcE
Left_Turn has quit [Ping timeout: 255 seconds]
Left_Turn has joined #osdev
<moon-child>
fuck fearless concurrency. I want fearless sequentiality
<moon-child>
only real bitches can do one thing and then do another thing
<mjg>
what rust does provide is fearless webdev
<mjg>
but you already get that with node.js
<zid>
programs that do a horrendous out of order mess make me sick
<Cindy>
out of order instructions
<Cindy>
why should that be a thing?
<GeDaMo>
Performance
<mjg>
but if i wanted things in different order i would have written them that way1111
<heat>
EYE
<heat>
TANI
<heat>
UM
<mjg>
peak-fuckin-webdev was when a fucking guy tried to equate one step of an algorithm with one cycle of a cpu
<mjg>
as i said a few times, not everyone should be programming
<heat>
peak-fuckin-unixdev is when that guy estimated clang's page fault performance in 2023 using a 1990s sparcstation's performance
<mjg>
that was peak geezer
<mjg>
i would not put it on unix dev specifically
<Ermine>
> not everyone should be programming - true
dude12312414 has quit [Quit: THE RAM IS TOO DAMN HIGH]
pebble has joined #osdev
bauen1 has joined #osdev
zetef has quit [Ping timeout: 264 seconds]
srjek has joined #osdev
Mutabah has quit [Ping timeout: 260 seconds]
roper has quit [Quit: leaving]
Matt|home has joined #osdev
eddof13 has joined #osdev
Mutabah has joined #osdev
<kof123>
> while we're discussing this NERD SHIT someone at the protobuf company added an interactive cat page with meows
<kof123>
i prefer: oct. 15 2012 107th Anniversary of Little Nemo in Slumberland - Google Doodles > This dreamy, non-linear tale tells the story of Nemo, the princess, and their allies as they try to stop the Emperor of Sol and the Guardian of Dawn
<kof123>
dreamen, plural :D
Gurkenglas59 is now known as Gurkenglas
Gurkenglas has quit [Quit: Client closed]
Gurkenglas has joined #osdev
xenos1984 has quit [Ping timeout: 246 seconds]
xenos1984 has joined #osdev
blockhead has joined #osdev
xenos1984 has quit [Ping timeout: 256 seconds]
<ghostbuster>
does anyone write games that run in pure uefi? wondering if i could port something simple like `sl` (the train animation)
xenos1984 has joined #osdev
<kof123>
> moon-buggy(6) - Linux man page - Die.net linux.die.net › man › moon-buggy
<geist>
nyancat for uefi
<heat>
sure you can
<heat>
i really doubt the UEFI console emulates all the VT100 escape codes for that
<heat>
for sl... maybe? i don't know if it sticks to ANSI escapes, but I also don't know how complete UEFI's console emulation is (i suspect it's not very complete)