<bslsk05>
'Open the Pod bay doors, please, HAL.' by batmanmmv (00:03:06)
Arthuria has quit [Ping timeout: 260 seconds]
gog has quit [Ping timeout: 260 seconds]
dude12312414 has quit [Quit: THE RAM IS TOO DAMN HIGH]
xFCFFDFFFFEFFFAF has quit [Ping timeout: 260 seconds]
rpnx has joined #osdev
goliath has joined #osdev
volum has quit [Ping timeout: 250 seconds]
PapaFrog has quit [Ping timeout: 252 seconds]
PapaFrog has joined #osdev
rpnx has quit [Ping timeout: 268 seconds]
rpnx has joined #osdev
warlock has joined #osdev
rpnx has quit [Ping timeout: 256 seconds]
\Test_User has quit [Quit: e]
\Test_User has joined #osdev
\Test_User has quit [Quit: \Test_User]
\Test_User has joined #osdev
rpnx has joined #osdev
gbowne1_ has joined #osdev
gbowne1 has quit [Ping timeout: 252 seconds]
rpnx has quit [Ping timeout: 268 seconds]
rpnx has joined #osdev
rpnx has quit [Ping timeout: 240 seconds]
spareproject has joined #osdev
\Test_User has quit [Quit: \Test_User]
azonenberg has joined #osdev
<azonenberg>
Hmmmmm
\Test_User has joined #osdev
<azonenberg>
Is the ELF program header table required to be at the beginning of the file?
<azonenberg>
more specifically, is it required to be before any program header content
<azonenberg>
I'm trying to think about how plausible it would be to create a streaming ELF parser that handles the file in a fly-by fashion without being able to rewind
<azonenberg>
i guess the better question is, is gnu ld ever going to output a binary that doesnt do this :p
<azonenberg>
(since i'm only going to be processing binaries i generated)
rpnx has joined #osdev
alexander has quit [Quit: ZNC 1.8.2+deb3.1 - https://znc.in]
<kazinsal>
I'm not 100% sure about "required" -- you'd need to do a deep dive into the SysV ABI specs for that -- but I don't think I've ever seen an ELF that wasn't like that
\Test_User has quit [Quit: \Test_User]
<azonenberg>
As in, as long as I have error detection to refuse to process a file constructed any other way
<azonenberg>
I should not have any problems with my linker making a file like that
alexander has joined #osdev
rpnx has quit [Ping timeout: 256 seconds]
<azonenberg>
Basically I'm thinking about container formats for firmware flashing on an embedded device, ELF is what my linker output natively
\Test_User has joined #osdev
<azonenberg>
i want to be able to just firehose the ELF at the embedded processor and have it either write each byte of data to flash or discard it, based on what it saw earlier in the file
<azonenberg>
but it doesnt have enough ram to store the whole image
<azonenberg>
i can cache some offsets from the header so i know how to interpret future data but i can't e.g. go back and be like "oh wait i wanted to read that section again"
joe9 has quit [Ping timeout: 268 seconds]
\Test_User has quit [Quit: \Test_User]
\Test_User has joined #osdev
rpnx has joined #osdev
rpnx has quit [Ping timeout: 256 seconds]
joe9 has joined #osdev
volum has joined #osdev
rpnx has joined #osdev
rpnx has quit [Ping timeout: 256 seconds]
gbowne1_ has quit [Quit: Leaving]
rpnx has joined #osdev
rpnx has quit [Ping timeout: 245 seconds]
<GreaseMonkey>
welp, discovered some good news and bad news about pre-PCI PnP
<GreaseMonkey>
the good news is the BIOS can do stuff for you and even lists off a bunch of stuff you can potentially probe
<GreaseMonkey>
the bad news is it barely helps you when you need to get information out of ISA-PnP cards, you *still* have to do the "send the key" song and dance
<GreaseMonkey>
alternatively one could feasibly go through the ESCD system from the PnP BIOS... but i haven't looked into the spec properly
<kazinsal>
yep, ISA PNP is a fuck
pebble has joined #osdev
<kazinsal>
insert that old clip of billg plugging a card into a Windows 98 prototype and the system immediately bluescreening
<zid`>
but what about the dmi pool data!?
rpnx has joined #osdev
rpnx has quit [Ping timeout: 256 seconds]
Burgundy has joined #osdev
<nikolapdp>
kek
rpnx has joined #osdev
vai has quit [Remote host closed the connection]
rpnx has quit [Ping timeout: 268 seconds]
Left_Turn has joined #osdev
bauen1 has quit [Ping timeout: 256 seconds]
gog has joined #osdev
bauen1 has joined #osdev
rpnx has joined #osdev
rpnx has quit [Ping timeout: 245 seconds]
GeDaMo has joined #osdev
<geist>
yeah i dont remember isa pnp being that widespread before pci came and replaced it
volum has quit [Quit: Client closed]
<clever>
have you heard about how the amiga pnp worked?
<clever>
zorro2 i think it was called?
<GreaseMonkey>
unfortunately, it was *too* widespread for me to avoid it in this instance
<GreaseMonkey>
PCI is *waaaaay* nicer
<GreaseMonkey>
amiga pnp, uhh i read a bit about "AUTOCONFIG"
<GreaseMonkey>
but that was some time ago and i forget the exact details
rpnx has joined #osdev
chiselfuse has quit [Remote host closed the connection]
chiselfuse has joined #osdev
<GreaseMonkey>
out of curiosity has anyone tried making something with FreeRTOS? i'm tempted to toy with it on my DOS rig, and apparently it works in real mode and even specifically has been tested w/ openwatcom
rpnx has quit [Ping timeout: 240 seconds]
<GreaseMonkey>
it was used at my previous job but i never dug deeply into it
<GreaseMonkey>
(despite the fact that i managed to insert a custom bootup handler for debugging purposes which could capture the old stack pointer and the stack... but that's before FreeRTOS is even set up)
rpnx has joined #osdev
netbsduser` has joined #osdev
rpnx has quit [Ping timeout: 268 seconds]
leg7 has joined #osdev
<leg7>
Hello, should I use volatile for IO port addresses and MMIO?
<leg7>
the C keyword that is
<kazinsal>
volatile tells the compiler that it should not make assumptions as to the value of a variable/pointer
<zid`>
io port addresses aren't in memory so you can't declare any storage that refers to them
<zid`>
but yes for mmio
<leg7>
I have some u16 variables that refer to their io address. for example `u16 data_port = 0x60`
<zid`>
volatile int a; just means that it will reload the value of 'a' every time it gets accessed, rather than caching it
<kazinsal>
x86 port I/O is not memory so the compiler doesn't really need to know about it'
<kazinsal>
it's functionally just data that appears to the compiler
<zid`>
so if(a > 5) { if(a == 6) { blah(); }} isn't allowed to compile to if(a == 6) blah();, it *must* do both accesses
<kazinsal>
MMIO is where you need volatiles because the compiler doesn't know that the MMIO region is MMIO
<zid`>
but that means nothing for your u16 data_port
<zid`>
because it's just.. a number in a variable
<zid`>
not an access to an io port
<leg7>
ok thanks guys I get it now
rpnx has joined #osdev
<kazinsal>
pointers to volatile addresses means that the contents of those addresses could be changed by something the compiler cannot predict
<netbsduser`>
leg7: it sounds like you want -I/path/to/kernel
<leg7>
yeah but doesn't that allow to just do something like `#include <io.h>`
<netbsduser`>
no
<netbsduser`>
it means essentially that if you #include <io.h> then GCC will try appending that to /path/to/kernel and try to open it, if it fails, it's not included
<netbsduser`>
it will not recursively open every subdirectory and try to append it there
<leg7>
huh really? because I use -I/path/to/libc/ and I can just do `#include <string.h>` in my code
<leg7>
even though `string.h` is in the `string` subdirectory
<netbsduser`>
probably because you are including it from string.c (which is in the same folder, so GCC searches there) or because you have not told GCC to exclude system includes
<bslsk05>
github.com: LigmaOS/Makefile at 8de6cd33252fb5efc451fbd676de49dfd42cd401 · leg7/LigmaOS · GitHub
<leg7>
ok
<netbsduser`>
one would assume i686-elf-gcc is not going to try anyway but you never know
<leg7>
well know compile fails because it can't find stddef.h
<leg7>
now*
srjek has joined #osdev
xFCFFDFFFFEFFFAF has joined #osdev
<leg7>
yeah ok even if I include stdlib.h it works even though I don't have that file so it must've been using system directory include
<leg7>
I read somewhere on osdev that even if you are freestanding and don't use system include you should be able to have access to stddef.h because it's bundled with the compiler
<leg7>
but that is clearly not working so maybe that statement is not true anymore
xFCFFDFFFFEFFFAF has quit [Remote host closed the connection]
rpnx has joined #osdev
rpnx has quit [Ping timeout: 252 seconds]
rpnx has joined #osdev
xFCFFDFFFFEFFFAF has joined #osdev
foudfou has quit [Remote host closed the connection]
foudfou has joined #osdev
slow99 has quit [Quit: slow99]
rpnx has quit [Ping timeout: 255 seconds]
slow99 has joined #osdev
rpnx has joined #osdev
rpnx has quit [Ping timeout: 272 seconds]
chiselfuse has quit [Remote host closed the connection]
chiselfuse has joined #osdev
rpnx has joined #osdev
rpnx has quit [Ping timeout: 245 seconds]
edr has joined #osdev
heat has joined #osdev
rpnx has joined #osdev
rpnx has quit [Ping timeout: 246 seconds]
<netbsduser`>
heat: about your radix tree page cache
<netbsduser`>
are you making use of RCU in it yet?
<heat>
not yet, no
<heat>
there are some things i need to think through
<heat>
mainly the use of marks and traversal based on marks
rpnx has joined #osdev
<netbsduser`>
i have been reforming my VM and i was deciding between a radix tree or something along the lines of (per-object) x number of rb trees (picked by hash of virtual address, or perhaps hash of virtual address shifted right enough so that e.g. 16 consecutive pages share the same hash - for ease of readahead)
<heat>
radix is most definitely optimal here
<heat>
rb trees are cache-and-branch-predictor pessimal
<heat>
and you can't really RCU them
<heat>
and they'll have a worst branch factor than radix
<netbsduser`>
the merit of the trees would mostly be the avoidance of having to allocate any extrusive structure (i could squeeze the linkage into my analogue of the page struct), but it's not a merit i am particularly attached to, so i am likely to use the same radix trees as i do for anonymous objects
joe9 has quit [Quit: leaving]
<heat>
the really powerful stuff about (linux style) radix trees is the ability to mark entries with specific info
<netbsduser`>
the potential of using RCU here is quite appealing because i don't want to get lost in outrageously complicated locking protocols to scale the page cache
<heat>
if i have dirty entries i'll mark the entries in the radix tree dirty, then on writeback finding what's dirty is super easy and efficient
<netbsduser`>
a sensible approach, i think
rpnx has quit [Ping timeout: 252 seconds]
<netbsduser`>
i was previously keeping dirtiness information in the page struct and i had queue linkage for writeback (as well as queue linkage for the active/inactive page queues) in the same. it wasn't ideal, writeback for files should be directed at a higher level
chiselfuse has quit [Ping timeout: 260 seconds]
<heat>
you probably still want to keep some dirtiness info in the page struct
<heat>
if you need to clean pages while reclaiming
<heat>
or trivially finding out if a page is dirty for some reason
<netbsduser`>
i still need it for anonymous and in general for good paging dynamics
chiselfuse has joined #osdev
Arthuria has joined #osdev
<netbsduser`>
now instead of an inactive queue which may contain dirty pages there is also an inactive-&-dirty queue and after writeback the page makes it to the inactive queue
<netbsduser`>
i will need to think a little about how the lockless lookup will work but i reckon it might actually be relatively straightforward
m257 has joined #osdev
<netbsduser`>
RCU on the radix tree while for the page struct you already have type-stability of the object
<netbsduser`>
at the moment i would still need to acquire the page queues lock after finding the potential page, because just speculatively raising the refcnt of the page will confuse other code. but i can take a close look at that later
<netbsduser`>
and even with that proviso the lookup itself being lockless in the happy case is a win
rpnx has joined #osdev
<heat>
if you can't take a ref without taking a lock it's a bit broken tbh
<netbsduser`>
it's because there would suddenly be a refcnt of > 0 which everything takes as evidence that the page is not on an inactive paging queue
rpnx has quit [Ping timeout: 255 seconds]
<netbsduser`>
it should be possible to work around i just haven't tried yet
<heat>
linux uses flags to know the lru list its on, if any
<leg7>
Is there someplace I can find the documentation of the "8042" chip? The wiki says it's now part of the 82091AA chip but intel seems to have removed the documentation for their website. I found a datasheet on a 3rd party website but there is no mention of a PS/2 controller and it seems to be directed to electrical engineers
<leg7>
I still can't find what I was looking for though. I wanted to know if IO port 0x60 (data port) is an 8bit, 16 bit, or 32 bit port.
netbsduser` has quit [Remote host closed the connection]
netbsduser` has joined #osdev
<heat>
8
<leg7>
ok thanks
rpnx has joined #osdev
rpnx has quit [Ping timeout: 256 seconds]
<leg7>
I really wish C had namespaces
navi has joined #osdev
<zid`>
good news, it does
<zid`>
it's called prefix your symbol with the name of the file it came from
<zid`>
e1000_send
<netbsduser`>
why namespaces of all things
<netbsduser`>
they aren't particularly useful, it's not like they're proper namespaces as you'd find in common LISP
<leg7>
yeah that's what I do but it gets confusing and you can't do stuff like `using namespace`
<leg7>
it's just nice to have
<zid`>
netbsduser`: i've never understood it either
<netbsduser`>
we call it "#include "namespace.h"
<zid`>
as best as I can tell people just enable it then instantly turn it off anyway
<zid`>
in C++
<nikolapdp>
FEARLESS NAME CLASHING
<leg7>
well I don't like c++ namespaces
<leg7>
they suck
<zid`>
you know that someone has to *link* the thing that you make, right?
<heat>
c++ namespaces are fine?
<zid`>
the dll needs to include it as a name, msdn articles need to be named it, man pages etc
<heat>
they're e1000_send, but inside the language
<zid`>
so going nuts with namespacing means you need to do man cstd::string::strlen
<zid`>
instead of man strlen
<netbsduser`>
heat: it's not that they're bad they're just not much good though are they?
<zid`>
I've never seen a use for them at least
<heat>
i find them to be a-ok
<zid`>
seems to.. work though?
spare has joined #osdev
<heat>
like, everything in namespace X doesn't need to do X_send()
<netbsduser`>
in common lisp you can e.g. pass a namespace in to something to have it consume APIs from that namespace, perhaps you implemented another namespace providing the same APIs
<heat>
or X::send()
<heat>
or std::string::X::send()
<netbsduser`>
whereas in C++ i don't think you can pass a namespace as a template parameter, for instance
<heat>
you can also do using namespaces
<zid`>
"you can also turn it off" :P
<heat>
netbsduser` you can use classes with strictly static members
<netbsduser`>
really it seems to me that they are less-flexible form of classes with only static members
<netbsduser`>
heat: snap
<nikolapdp>
zid`, don't forget, heat writes in c
<zid`>
o yea
<zid`>
his sick perfect saef subset
<heat>
netbsduser`: that's mostly what they are yeah
<heat>
but you can't do using std::string; and have all its members imported to your global namespace
<leg7>
also you don't really know what the "namespace" is in your function/var name
<heat>
sure you do
<leg7>
for x86_32_outb is it x86_32::outb or x86::32::outb?
rpnx has joined #osdev
<heat>
namespace 32 is a syntax error
<leg7>
ok
<leg7>
> c++ namespaces
<heat>
what
<leg7>
anyways it's just nitpicking
<leg7>
ofc namespaces aren't a new powerful feature
<leg7>
it's just something I think would be nice to have
<heat>
C++ is full of nice-to-haves that C people like to handroll every time they need something
<heat>
the gang handrolls a shitty linked list
<leg7>
that's a different story
<heat>
no, it's the same story
<heat>
namespaces??? who needs that when you can handroll your own prefix scheme!
<leg7>
well one is a language feature while the other is a standard library
zxrom_ has joined #osdev
<heat>
the spec does not know what a library is
<heat>
could be all language features as long as it compiles according to the spec
<heat>
the C++ spec does not specify what happens when #include <map> happens, only that std::map pops up
zxrom has quit [Ping timeout: 245 seconds]
rpnx has quit [Ping timeout: 260 seconds]
<leg7>
have you tried stepping through a standard library call?
<leg7>
it's impossible
<leg7>
maybe you're right and it's the same problem
<leg7>
but I think the c++ standard library isn't perfect and sometimes you need to roll out your own thing
<leg7>
although most times their shit is enough
<heat>
c++ is most definitely very flawed yes
stolen has quit [Quit: Connection closed for inactivity]
<bslsk05>
pastebin.com: use std::collections::HashMap;use std::io::{self};use std::process::Command; - Pastebin.com
vin has joined #osdev
<mjg>
i'm stumped by trying to add an object to a hashmap
<mjg>
don't tell me one needs to add reference counting around it
<vin>
Can the start address of an application heap change during its runtime? 562ad2b84000-562ad2ba5000 rw-p 00000000 00:00 0 [heap]
<netbsduser`>
mjg: rust?
rpnx has joined #osdev
<FireFly>
mjg: I think your Child is stack-allocated, so it makes sense you can't keep a reference to it after the end of the function
<FireFly>
althoug my rust-fu isn't the strongest--I'd suggest asking in ##rust :p
<mjg>
it may be
<mjg>
how do i heap-allocate it then
<FireFly>
I don't see a reason why you'd need to refcount, but maybe Box<> it or so?
<FireFly>
heap allocations is what Box is for
<m257>
when in doubt kill the child
<zid`>
s/when in doubt//
rpnx has quit [Ping timeout: 256 seconds]
<gog>
kill
leg7 has quit [Ping timeout: 245 seconds]
heat has quit [Quit: Client closed]
<mjg>
FireFly: took the plunge and asked on the forums :X
rpnx has joined #osdev
rpnx has quit [Ping timeout: 268 seconds]
<azonenberg>
kazinsal: ISA pnp??
heat has joined #osdev
<azonenberg>
i... did not even realize that existed
<azonenberg>
i thought ISA was entirely manual config and PnP landed when PCI happened
<azonenberg>
But the only ISA-based systems I've used were built/configured by someone else (thankfully)
<heat>
you still use some form of PnP with ACPI
<heat>
ACPI device ids are PNP ids
<azonenberg>
yeah i thought ACPI was newer than ISA
<heat>
it is yeah
<heat>
mjg: you learning rust?!
<azonenberg>
i thought ACPI and PCI both landed as part of a broader "stop manually configuring irqs and memory ranges of peripherals by hand" PC architecture revamp
<heat>
i need to learn more rust then
<heat>
i can't be left behind
<azonenberg>
lol dont we all
<heat>
shhh don't wake up the rust haters
<azonenberg>
everywhere i look demand for rust skills is exceeding staff
<heat>
azonenberg: naw it was a phased thing
<heat>
first you had PNP, then PCI I believe, then ACPI
<nikolapdp>
no we don't
<azonenberg>
personally, i'm not a fan of it. like, i like the idea of a safe systems programming language
<azonenberg>
i just dont think rust is it
<azonenberg>
where's my memory safe language with multiple inheritance and proper OO support
<heat>
rust looks pretty much it
<heat>
lol use C++
<nikolapdp>
safe language -> c++
<heat>
not memory safe but with enough glue and wrappers it's C++
<nikolapdp>
lollers
<azonenberg>
yeah thats the thing, i want C++ with a little guardian angel that saves me from doing stupid things
<heat>
glue and wrappers azonenberg, glue and wrappers
<azonenberg>
not a padded cell where memory thoughtcrime is impossible
<azonenberg>
(which is what rust feels like)
<mjg>
heat: no, i stick to php
<heat>
omg RUST IS LITERALLY 1984!
<azonenberg>
I still write all my low level stuff in c++ lol
<heat>
the ministry of the borrow checker and the ministry of concurrency
<azonenberg>
lolol
<azonenberg>
although for PC-based (vs embedded) i'm starting to use more shared_ptr's and such and it wouldnt surprise me if i end up with architectures that would feel at home in rust
<netbsduser`>
azonenberg: sometimes you need to take control of things yourself
<azonenberg>
i've been watching this project with interest
<mjg>
concerning
<heat>
a Real Kernel uses and abuses refcounts
<Ermine>
unsafety is safety
<heat>
so yes, maybe shared_ptr
<netbsduser`>
if you are using weird lifespans for everything, RCU everywhere, all that stuff, then rust will be an obstacle
<mjg>
maybe heavily_shared_ptr
<heat>
although shared_ptr isn't my favourite, in my kernel i have a refcountable classes inherit from
<heat>
when i don't use special refcount handling myself
<azonenberg>
i have a large (~200kloc) C++ codebase now that has a bunch of homegrown refcounting that is proving to be a huge pain and i'm in the process of transitioning to use shared_ptr
<netbsduser`>
it's very pleasant
<azonenberg>
(HPC application not an OS)
<netbsduser`>
i end up doing tricks with almost everything so i wouldn't immediately be able to benefit hugely from shared_ptr, but even then there are odd places it would be helpful
<heat>
homegrown refcounting is aight if it's correct
<heat>
it many cases it's needed (what does refcount = 0 mean in shared_ptr??)
<netbsduser`>
destruction
<Ermine>
aight
<gog>
but what if you need it later
<heat>
netbsduser` i know, that's the problem ;)
* Ermine
gives gog a piece of cheese
<heat>
wow the web.libera.chat ;) emoji looks like i'm DTF
<azonenberg>
Yeah lol, thats one of the problems i'm facing wrt refcounts too
rpnx has joined #osdev
<netbsduser`>
i have a funny refcount for pages (0 = either this is now deleted and freed, or this is now on an inactive queue, depending on the use)
* gog
takes a reference to the cheese
<azonenberg>
in this case controlling multiple users of e.g. an oscilloscope channel that needs to turn off if nobody is using it
<bslsk05>
imgur.com: Imgur: The magic of the Internet
<azonenberg>
but i still need the channel object to exist since the parent scope has to be able to ref it
<azonenberg>
but at the same time, say a math function needs to actually get deleted when refcount hits zero
<netbsduser`>
and on vnodes the 1-to-0 refcount drop invokes a special fs routine (necessary to get the vnode out of an inode number to vnode cache, if the fs has one)
m257 has quit [Quit: Client closed]
<azonenberg>
i'm still trying to figure out the details, but i'm going to need some logic to handle the special case of a shared_ptr with only one ref or something probably. except it may not literally be one ref, this might be better done via separate tracking of who's consuming th edata
<azonenberg>
i.e. it's ok if you have a properties dialog open, that shouldn't keep the channel on
<azonenberg>
but a waveform view of the channel should
<heat>
yep
<azonenberg>
anyway in general i think ultimately i need to separate object lifetime t racking from walking loads in the filter graph
<heat>
in no way should object destruction be the thing that triggers the device to shutdown
<azonenberg>
there's other parts of the code where i more generally need to track consumers of data
<azonenberg>
so i can e.g. have a math function omit a time-consuming "generate X value" operation if that port in the flow graph is floating and not driving any loads
<netbsduser`>
yeah, one has to be able to cope with a dissociation of object lifespan from the lifespan of what it represents
<netbsduser`>
usb flash drives for one
<azonenberg>
yeah in my case the idea is, i want to turn off the channel in hardware if nobody is using its output
<heat>
atomic<unsigned int> active_users;
<azonenberg>
because that uses resources like sample memory and network bandwidth that i may want to use for other stuff
<heat>
if (active_users.sub_fetch(1, release) == 0) turn_off()
<azonenberg>
(e.g. some scopes have higher sample rate available if only a few channels are in use)
<azonenberg>
heat: That's what i have now but it's using the refcount
<heat>
might need a lock there dunno
<azonenberg>
Which is where things went sideways
<heat>
yeah but the refcount is not really what you should be using
<azonenberg>
and there's actually two kinds of refcount implementing the same base class
<azonenberg>
one for filters which are deleted when it hits zero
<azonenberg>
one for instrument channels which turn off
<azonenberg>
the confusion of these two proved to be a terrible architectural decision
<azonenberg>
and one i'm now paying for in spades :p
<heat>
yay tech debt
rpnx has quit [Ping timeout: 256 seconds]
<heat>
anyway RUST IS LITERALLY 1984 ANTI FREEDOM
<heat>
OFC JOE BIDEN ORDERS US TO USE RUST
<heat>
YEEHAW NEVER TAKING MY C AWAY
<Ermine>
who's Rust big brother?
m257 has joined #osdev
<heat>
Ministry of Borrowing
<azonenberg>
anyway, in general a broader thing i am looking into is how to get high performance, generally low level code (marshaling and unmarshaling raw sample data off sockets, interfacing to GPUs, maybe RDMA in the future, etc) to be as reliable and memory safe as possible
<Ermine>
Graydon Hoare is the Big Brother
<azonenberg>
while still being practical to develop without what feels like the unreasonable constraints of rust
<heat>
rust has perfectly reasonable constraints for most kinds of programs
<azonenberg>
something about "RoCE -> Vulkan compute shader" doesn't seem like a flow rust would handle well :)
<heat>
most people struggle to lock and unlock their mutexes
<heat>
they don't care about mega-great RCU unsafe code
<azonenberg>
(a lot of folks seem to be having issues with it in gamedev for similar reasons)
<azonenberg>
And at least from a transition perspective i think there's a lot that can be done with C++ to head in that direction, and perhaps make a future rust port easier
<azonenberg>
without a complete rewrite yet
<Ermine>
Anyway, rust thoughtpolice was already dispatced
<azonenberg>
Lol
<azonenberg>
anyway, back to my original question from before, it sounds like it should be possible to do what i want
<azonenberg>
namely, a single-pass ELF loader that can't rewind the file
<azonenberg>
and just reads the elf header, program header table, and each section in sequence
<heat>
i don't think that's possible, no
<azonenberg>
as long as nobody goes out of their way to craft a pathological file in which e_phoff is not pointing right after the elf header
<heat>
i can stick program headers anywhere
<azonenberg>
Yes, it's possible to create such a file
<azonenberg>
that's not what i'm asking :)
<heat>
it's a valid ELF file
<azonenberg>
yeah. but there's plenty of tools that have limits more strict than those imposed by the file format
<heat>
you can get all kinds of limited simple ELF loaders but they don't load all the ELFs
<azonenberg>
e.g. imagej won't work with images that have more than 2^31 pixels in them
<azonenberg>
even if the format supports bigger
<heat>
sure
<azonenberg>
my application here is firmware updating for an embedded device i control
<azonenberg>
so i dont think its unreasonable to have a constraint that your program header table has to be before your sections and simply refuse to flash any file that doesn't comply with that requirement
<heat>
yeah that's fine
<azonenberg>
(any linker i've ever worked with will do that, you'd have to go out of your way to make a bizarre-but-technically-spec-legal file to do that)
<azonenberg>
to break that*
vdamewood has quit [Quit: Life beckons]
<azonenberg>
i.e. there's no way that i or anyone who forks my firmware repo on github and builds using my build flow is going to accidentally generate such a technically-not-malformed elf
<azonenberg>
and if they somehow throw one at it, my bootloader is just gonna refuse to touch it :p
<azonenberg>
(I don't see much of an alternative here given that i want to keep elf metadata for things like secure boot signatures in the future, and i don't have enough RAM to store the entire image before writing to flash)
<Ermine>
Microsoft drm is claimed to be hacked
<heat>
ELF doesn't have a signing mechanism
<heat>
it's brilliant
<Ermine>
you can add a section containing a signature, can't you
<azonenberg>
Yes, that's the idea
<heat>
you could but it's non-standard and already implies you have to parse a bunch of untrusted ELF
<heat>
the linux kernel module signing mechanism is seriously amateurish too
<azonenberg>
Ultimately what I want to do is define a new non-loadable section containing a curve25519 signature over metadata and contents of all PT_LOAD sections
<azonenberg>
that is filled out to a dummy value in my linker script then a post-link sign step populates with actual content
<azonenberg>
The goal is to have a freedom-preserving, safe update flow where the system defaults to trusting my signing key but given ssh admin access you can revoke mine and/or add your own root
<azonenberg>
possibly with a jumper you have to toggle or something to enforce physical access when doing this
voidah has joined #osdev
<azonenberg>
it wont be full secure boot, at least for now (no boot-time signature checks only when loading new images)
<azonenberg>
the main goal is to protect against tampered downloads
<azonenberg>
milestone 1 is going to be basic flashing with no integrity checks other than those provided by the SFTP protocol itself, i.e. you trust that the ELF is fine and only guard against tampering between the admin's PC and the device
<azonenberg>
milestone 2 will verify the signature in the bootloader during the flashing process before letting it execute
<azonenberg>
milestone 3 will be boot time checks and may or may not ever happen
<azonenberg>
at least for the FPGA i will likely only ever use the CRC in the bitstream, i don't see going full secure boot there
<azonenberg>
phsical attacks are outside my threat model
<azonenberg>
I think it should be possible to implement this update model reasonably safely with a few basic sanity checks on the ELF, for example "no loadable section is allowed to overlap the bootloader or SFR regions, it can only be application flash"
<azonenberg>
a lot of potential parsing footguns will be eliminated by the fact that i'm basically only ever parsing the program header table
foudfou has quit [Remote host closed the connection]
foudfou has joined #osdev
rpnx has joined #osdev
rpnx has quit [Ping timeout: 268 seconds]
rpnx has joined #osdev
rpnx has quit [Ping timeout: 255 seconds]
xenos1984 has quit [Ping timeout: 240 seconds]
xenos1984 has joined #osdev
<Bitweasil>
*grumbles* ARMv8 has an extension that allows for load/store exclusive to work with virtual addresses as well as physical. And I cannot for the life of me find it. :/
<bslsk05>
github.com: conc-map-bench/results/ReadHeavy.std.throughput.svg at master · xacrimon/conc-map-bench · GitHub
<mjg>
approximately 10 mop/s with 1 thread to just above 100 with 32
<mjg>
fucking terrible
dude12312414 has quit [Remote host closed the connection]
<mjg>
well they have another one which does a little better, but they are all clearly bogged down by cacheline pingpong
dude12312414 has joined #osdev
<mjg>
hw is 56 cores
gorgonical has joined #osdev
<gorgonical>
hello to the kernal developers today
<gorgonical>
and to those like me who merely pretend to
<FireFly>
yes hi pretender here
<gorgonical>
I wasted all of friday just about trying to figure out how to allow userspace to provide scatter-gather buffers to hardware
<gorgonical>
But then alignment concerns killed that idea and I switched to hinted mmapping instead
xenos1984 has joined #osdev
<gorgonical>
riddle me this: what's the point of posix_memalign if implicitly virtual memory may not be aligned the same as the underlying physical memory
<gorgonical>
I mean sure, having page aligned memory is useful
Arthuria has joined #osdev
<gorgonical>
But you can align to an arbitrary size with posix_memalign, which really is not likely to be supported by the underlying os I would reckon
Bonstra_ has quit [Quit: Pouf c'est tout !]
Bonstra has joined #osdev
zxrom_ has quit [Quit: Leaving]
rpnx has joined #osdev
<netbsduser`>
gorgonical: it's rather simple really
<netbsduser`>
you can turn a range of user address space into a scatter gather list if it is aligned appropriately
dude12312414 has quit [Quit: THE RAM IS TOO DAMN HIGH]
Arthuria has quit [Ping timeout: 260 seconds]
<Bitweasil>
gorgonical, there are requirements for other load/stores too - most of the vector load/stores are either required to be aligned natively, or are massively slower if they're not (depends on the hardware). And while virtual memory will muck with page locations, it won't mess with sub-page alignments.
<Bitweasil>
Something aligned to 8 bytes will be aligned to 8 bytes, no matter how the page is mapped.
<Bitweasil>
It's only when you get larger than page size that it starts being weird.
zxrom has joined #osdev
<Bitweasil>
And at that point, it really only matters if you're doing DMA buffers or stuff, or care a lot about making sure you're mapped on large pages.
leg7 has quit [Remote host closed the connection]
<gorgonical>
Bitweasil: indeed regarding the page size you are fully right
<gorgonical>
I am in fact concerned with device dma/iommus, so it sort of puzzles me there
<gorgonical>
I can request 4MB aligned allocs which it gives me, but the underlying physical memory assigned isn't aligned the same way (perhaps it ought to be)
<Bitweasil>
You'll want to do some reading on the userland device driver support and interfaces.
<Bitweasil>
I've never dealt with them, but in general, Linux at least will not guarantee that userspace pages don't even move around in memory, unless they're pinned (which... I don't recall if there's even support for that in userspace).
<gorgonical>
I think currently the underlying physical memory will only be page aligned, but even if I changed page granularity to 2mb it wouldn't guarantee it here
<Bitweasil>
But userspace drivers are a thing, and they'll have to have solutions for this sort of problem.
m257 has quit [Ping timeout: 250 seconds]
<gorgonical>
that's a good idea. The mmap interface has some knobs and the dma_alloc (I think) does respect whatever the underlying device says it needs wrt alignment
<bslsk05>
www.kernel.org: The Userspace I/O HOWTO — The Linux Kernel documentation
srjek has quit [Ping timeout: 268 seconds]
_whitelogger has joined #osdev
<vin>
Can I limit the virtual address space range an application can use? So that malloc/mmap must only allocate/map in a certain range of addresses. I will keep it big enough but I don't want 2^47 different addresses
<geist>
sure, that's entirely a software thing
<vin>
is there a syscall for that in linux geist?
<geist>
oh you mean linux? hell if i know
<geist>
i thought you were writing an OS
<vin>
No sorry, I making changes to linux
spare has joined #osdev
valerius_ has quit [Ping timeout: 264 seconds]
rpnx has quit [Ping timeout: 260 seconds]
voidah has quit [Ping timeout: 260 seconds]
Terlisimo has quit [Quit: Connection reset by beer]
goliath has joined #osdev
vdamewood has joined #osdev
Terlisimo has joined #osdev
carbonfiber has quit [Quit: Connection closed for inactivity]
<Bitweasil>
What architecture? There are plenty of architectures that can be configured to use less than 48 bits or 56 bits of VA space (ARMv8 being one), so I'd look into those mechanisms.
Yoofie8 has joined #osdev
Yoofie has quit [Ping timeout: 245 seconds]
Yoofie8 is now known as Yoofie
spare has quit [Remote host closed the connection]
<heat>
vin: nope, in general not possible
<heat>
you have MAP_32BIT at best
<heat>
the thing Bitweasil is referring to is wrt 48 vs 57-bit support, but 57-bit support is rare enough to begin with
<heat>
geist: this is where the beOS vm design shines isn't it?
<heat>
where you can create a sub-vm area and then sub-vm areas from that?
<heat>
>Note: although several shortcuts have been built into the makefile, the nonseparate I/D make occasionally runs out or space while recompiling the kernel. If this happens, just restart it and it will generally make it through the second time
<heat>
lol
zetef has joined #osdev
<Bitweasil>
heat, ARMv8 supports a bunch of sub-48-bit VA sizes too. I'm not sure if Linux has support for them or not.
<heat>
they support everything
<heat>
but i sincerely doubt they don't give out 48-bit VAs by default
<heat>
that was purely a compat decision by the x86 folks (and then spread to arm64 and riscv64) because of processes using the top bits for their own stuff
<GeDaMo>
vin: apparently Linux has something called control groups (cgroups) which can be used to limit process memory
<vin>
Yes GeDaMo that does not limit per process address range though. Just the amount of memory per process or bunch of processes
<vin>
heat: yes 57-bit for 5-level paging.
<nortti>
< heat> >Note: although several shortcuts have been built into the makefile, the nonseparate I/D make occasionally runs out or space while recompiling the kernel. If this happens, just restart it and it will generally make it through the second time ← what're you reading? some 16-bit unix port's notes?
<bslsk05>
rust-lang/rust - Empowering everyone to build reliable and efficient software. (12007 forks/93042 stargazers/NOASSERTION)
<heat>
it is not kernel code
<nikolapdp>
sure is
<nikolapdp>
they took stdlib and adapted it (poorly)
<heat>
no
<nikolapdp>
sure
<heat>
if you saw the presentation i linked she goes into detail on failure and failure allocating and pre-allocating and all of the differences wrt userspace rust
Left_Turn has quit [Read error: Connection reset by peer]
<Ermine>
panic!
<nikolapdp>
panic!
<mjg>
heat, do you have a copy trait?
<heat>
no
<heat>
i only have the trait of being AWESOMEANDBASED
<mjg>
sounds like mismatched bounds
<Ermine>
I didn't advance past the third chapter of rust book