<Mutabah>
Despite how long I've been doing osdev, I've only just started doing real filesystem write support... and damn is it more complex than I would have thought. Lots of potential race condition points (... maybe my fault for starting with FAT)
[itchyjunk] has quit [Read error: Connection reset by peer]
[itchyjunk] has joined #osdev
Gooberpatrol66 has joined #osdev
hmmmmm has quit [Remote host closed the connection]
hmmmm has joined #osdev
vdamewood has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
[itchyjunk] has quit [Ping timeout: 250 seconds]
Arthuria has joined #osdev
heat has quit [Remote host closed the connection]
heat has joined #osdev
epony has quit [Ping timeout: 268 seconds]
tiggster has joined #osdev
Arthuria has quit [Killed (NickServ (GHOST command used by Guest684531))]
Arthuria has joined #osdev
tiggster has quit [Ping timeout: 260 seconds]
heat has quit [Remote host closed the connection]
Arthuria has quit [Read error: Connection reset by peer]
Arthuria has joined #osdev
Arthuria has quit [Ping timeout: 260 seconds]
epony has joined #osdev
Arthuria has joined #osdev
mctpyt has quit [Ping timeout: 276 seconds]
bradd has joined #osdev
deflated8837 has joined #osdev
mctpyt has joined #osdev
Arthuria has quit [Remote host closed the connection]
xenos1984 has joined #osdev
bgs has joined #osdev
<klys>
mutabah, you have a reentrant or thread-safe FAT implementation?
<Mutabah>
That's the goal
<klys>
how do you handle multiple writers to the FAT table itself?
<Mutabah>
Locks individual sectors
<klys>
considering more than one FAT sector could be queued for writing theoretically
<Mutabah>
There's a block device chacing backend that only allows unique writing to a block
<klys>
suppose you have a user who is running an emulator such as qemu, you can resolve contention over a mounted filesystem?
<Mutabah>
what do you mean?
<Mutabah>
Someone using a hacky virtual FAT volume?
<klys>
suppose your user is emulating a pc and running your FAT implementation, and also has it mounted locally using your implementation
<Mutabah>
Oh, nope. The locking is within the driver only
<Mutabah>
Can't have multiple "machines" editing the volume
<klys>
would that be easier done if the FS in question were not FAT ?
<Mutabah>
Maybe. FAT's challenge is that file metadata is stored in the directory entry
<Mutabah>
so to update the size of a file, you need to find it in the dir again
<Mutabah>
... but maybe the file has been moved? maybe the directory has been defragmented?
<zid`>
plus sounds like you might end up with locking issues
<zid`>
lock(file) because of a write, then go to lock(dir)
<zid`>
but soemone else has the directory lock
<zid`>
so you need some weird invariant that you can't take a directory lock in order to take a file lock
<Mutabah>
Yeah, it's fun
<Mutabah>
I think I have a solution with taking the "open files" lock first, then taking the directory lock. This applies to `open` and `update_size` calls
<zid`>
just don't write any code that breaks the above invariant and you won't have to think about it
<zid`>
it'd be nice if there was a way to enforce it though
<Mutabah>
Lock ordering is fun :)
<Mutabah>
Thankfully, this is rust code - so it's basically the only thread safety invariant you need to worry about
<zid`>
I mean, you could enforce it with a bunch of sw overhead, but I'm not sure you can do it *without*
<zid`>
sans like, an ml linter
gbowne1 has quit [Quit: Leaving]
<klys>
sounds like a locking challenge that may exist at the same layer as media access control, thereby requiring a style of socket that really is a file.
<zid`>
that's the 'with sw overhead' version yes
<zid`>
struct fs_lock_state { /* opaque */ }; get_file_lock(&fs_lock_state, blah); where get_file_lock checks that you already have a directory lock, or whatever
<zid`>
by holding both locks
slidercrank has joined #osdev
<zid`>
turning it into a socket is just the microkernel version :P
<klys>
...until you have twenty hosers all on different timetables trying to lock the root directory
<klys>
so perhaps there is a way, though I think it would require some tracking as to who is available to assert a lock
<zid`>
You just need to pick invariants that.. actually work, which is the hard part
<klys>
contention may exist, test and set lock may not exist, the smallest block write is to a whole sector, yet this requires a functional network because there's no TSL
<klys>
without TSL, you could still assign each new writer to a safe space block, and thereby implement a collision detect mechanism.
<zid`>
Or
<zid`>
you could put a couple of quick invariants onto the locking
<klys>
is this a shared data object? how does it 'lock'?
<zid`>
filesystems are shared data yes
<zid`>
unless you're uniprocess
<klys>
I am supposing that there is not a uniprocess, but rather a dual mount which is shared even if both systems aren't otherwise networked.
<zid`>
nobody supports that it's insane
<zid`>
btw
<zid`>
it's also on the wrong layer
<klys>
the filesystem should typically rest on a higher layer than media access control, collision detect, etc. you may mean
<zid`>
lower
<zid`>
but yes
<zid`>
fs is basically a hw driver
<zid`>
nfs just looks like a random process issuing read() and write(), from its perspective
<zid`>
it just happens to be samba-daemon and not a local process
bnchs has quit [Remote host closed the connection]
GeDaMo has joined #osdev
bgs has quit [Remote host closed the connection]
hmmmm has quit [Remote host closed the connection]
vdamewood has joined #osdev
hmmmm has joined #osdev
<sakasama>
Are there any important modern architectures for which L1/L2/L3 cache line size aren't all the same?
<vdamewood>
Are there any importantmodern architectures at all?
<zid`>
amd64, the end
<sakasama>
That's an important question as well, but the emphasis of my question is on cache line size.
<zid`>
making l1/l2/l3 different sizes doesn't sound.. performant
<zid`>
and only performant chips *have* l1/l2/l3
<sakasama>
It could make sense to fetch larger lines into L2/L3, but... does anyone bother?
<zid`>
you'd just abstract that as 'prefetching'
<zid`>
and not as the lines being that size, and that lines get split into l1
<sakasama>
You could do that, yes.
<zid`>
and good news, intel/amd both do prefetching
<zid`>
so technically the answer is either 'no, nobody does that' or 'yes, intel and amd both do that but call it prefetching'
* sakasama
pours a glass of a murky liquor.
<sakasama>
I guess I can just ignore the distinction for now then.
bradd has quit [Ping timeout: 250 seconds]
<geist>
i think almost by definition the L1/L2/L3 caches aren't the same size
<geist>
oh cache line size
<geist>
yah i've seen a few
<geist>
usually the later L cache lines are larger
<geist>
IIRC intel even toyed with it a bit. I think core 1 or core 2 had some sort of larger cache line size in the L3 iirc
epony has quit [Remote host closed the connection]
<zid`>
yonah and kentsfield are both 64/64/64
<zid`>
no data for the penryn core 2 solo cpus (all 4 of them)
<geist>
for some reason i remember one of those in that era had 128 byte lines at some level
<geist>
but i also may be wrong
<zid`>
M1 has it apparently?
epony has joined #osdev
<geist>
oh possibly
<geist>
ARMv8 would totally allow for different cache line sizes at various levels
<moon-child>
as I recall maynard handley says re m1 'it's complicated but the cache lines are 64 bytes'. But forget the details
<moon-child>
intel l2 when you fetch a cache line always fetch the adjacent cache line by default, so you can kindaa think of it as having 128-byte cache lines
bradd has joined #osdev
Burgundy has joined #osdev
nyah has joined #osdev
Burgundy has quit [Ping timeout: 255 seconds]
slidercrank has quit [Remote host closed the connection]
slidercrank has joined #osdev
ketan has joined #osdev
Burgundy has joined #osdev
gog has joined #osdev
slidercrank has quit [Quit: Why not ask me about Sevastopol's safety protocols?]
heat has joined #osdev
<heat>
LONIX
<heat>
OPERATING SYSTEM KERNAL
heat has quit [Remote host closed the connection]
heat has joined #osdev
<sham1>
It's running on the Commodore now?
<heat>
yes i had to reboot into the KERNAL
Left_Turn has joined #osdev
<heat>
ChadGPT, what's __FBSDID for?
<heat>
i don't get it, all BSDs seem to have a similarly-cursed thing.
<zid`>
I'm going to one up commodore and make an operating system colonet
<zid`>
colonel*
goliath has joined #osdev
<heat>
"Use printf(3), not fputs(3), puts(3), putchar(3), whatever; it is faster and usually cleaner, not to mention avoiding stupid bugs." - style(9)
<heat>
all freebsd man pages should not be taken at face value
<zid`>
cout is superior
bradd has quit [Ping timeout: 250 seconds]
<zid`>
my style guide says to always use std::cout << std::hex << std::setfill('0') << std::setw(8) << x << std::dec << std::endl;
<heat>
preach
<heat>
global format modifiers are gr8
<Ermine>
heat: is that __FBSDID thingie on top of every c file in fbsd?
<heat>
yea
elastic_dog has quit [Read error: Connection reset by peer]
<heat>
i understand that on the very top they have those cvs marker things for bad version control or something, but __FBSDID is just a macro which gets used like __FBSDID("$FreeBSD$")
<Ermine>
I guess this is some RCS or CVS related bullshit. There's similar bullshit in NetBSD code
<zid`>
SVN is CVS done right, don't you know
Turn_Left has joined #osdev
<zid`>
That means it supports advanced features like commits
<heat>
thank you, i should've wikipedia'd but asking you was more fun
<heat>
and i would probably get a better answer anyway
<heat>
there are a bunch of typos in that article like "Russian" or "Ukrainian"
<heat>
they're not in the UN charter
<Ermine>
Oh no!
<Ermine>
Belarussian also
<Ermine>
Anyway, I've got interested too
<sortie>
You ever have one of those days when your OS will randomly lose 43 bytes of TCP data?
<sortie>
#define EAGAIN 43
<clever>
are you just adding the return code to the bytes transfered?
<sortie>
That's what userspace does yeah
<clever>
sortie: and if the return code is negative due to an error? :D
<GeDaMo>
Obviously you should be adding the abs of the return value :P
<clever>
lol
Left_Turn has joined #osdev
Turn_Left has quit [Ping timeout: 260 seconds]
<heat>
clever, the bug here is that sortix errors usually look like "return errno = EAGAIN, -1;" where this one was "return errno = EAGAIN;"
<heat>
and sortix does not do the standard thing of doing negative error codes like linux
<heat>
so EAGAIN was looking to userspace like it had sent $EAGAIN bytes of data
brynet has quit [Quit: leaving]
dude12312414 has joined #osdev
<clever>
ahhh
voidah has joined #osdev
bnchs has joined #osdev
<dh`>
negative error codes aren't standard, they're a foolish linuxism
<clever>
and libc also hides that from you
<clever>
it stores the codes in errno, and converts it to plain -1
<clever>
which means the kernel doesnt have to be aware of where errno is and thread local storage
dude12312414 has quit [Quit: THE RAM IS TOO DAMN HIGH]
xenos1984 has quit [Ping timeout: 265 seconds]
xenos1984 has joined #osdev
slidercrank has joined #osdev
brynet has joined #osdev
<sortie>
heat, what do you mean standard, being unable to return certain perfectly valid return values from system calls?
<clever>
sortie: how do you get error codes from kernel to userland?
<sortie>
Just use another register for errno
<clever>
ah, so its a wonky ABI with 2 return values
<sortie>
E.g. on 32-bit, I can return a 64-bit value in eax:edx and then errno in ecx
<clever>
that works
<sortie>
And then ecx is just copied to the thread local errno
<clever>
yep, same job libc is doing on linux, keeping TLS out of the kernels job
<clever>
but you can do negative return values, and return values with an error
<sortie>
Well the kernel is very aware of tls
<sortie>
It just doesn't know where errno is located and it doesn't need to
<clever>
exactly
<sortie>
Yeah I can return -2 and with errno = EINVAL
<clever>
no need for the kernel to go poking around in the userland TLS segment
<sortie>
It's also much slower
<clever>
let libc and the runtime linker figure it out
<sortie>
Would need to a secure copy out
<clever>
on some systems, there is kinda only 1 TLS var? and its a pointer into the regular heap?
<clever>
and i can see that being an exploit, if you point that into kernel space, and then expect the kernel to follow it
<heat>
sortie, where exactly do you need to return values in the upper 4096 range
<heat>
negative error codes are ideal and I don't see how you would not like them
<sortie>
Oh heat one day you will see the way of the Sortix
<heat>
it clobbers one less register, and you keep having easy and fast ways to detect errors
<sortie>
Means you need a temporary variable to capture and forward errors and handle them
<sortie>
errno is much easier since it's there
<sortie>
errno programming is much simpler everyone knows that
<sortie>
Except POSIX
qubasa has quit [Remote host closed the connection]
<heat>
i don't like errno
<heat>
it's a really weird kludge
<heat>
which got really kludgified further when threads came around
<heat>
there are also all those fun edge cases where functions may very well validly return -1 and then you need to do errno = 0; func(); and then read errno
<heat>
and ofc POSIX rolled around and added a bunch of functions which do not use errno
<heat>
for reasons
inegatives has joined #osdev
<heat>
so now there's no consistency even, just sadness
<sham1>
I don't like that errno has to be a macro
<heat>
does it?
<heat>
it's just allowed to be a macro
<sham1>
Yes
<sham1>
It's explicitly specified to be a macro. Of course you could do something like "extern _Thread_local int _errno; #define errno _errno" but still
<heat>
it's not
<heat>
" It is unspecified whether errno is a
<heat>
macro or an identifier declared with external linkage. If a macro definition is suppressed
<heat>
in order to access an actual object, or a program defines an identifier with the name
<heat>
errno, the behavior is undefined."
xenos1984 has quit [Ping timeout: 248 seconds]
<sham1>
Well then the spec is written in a confusing way where it contradicts its own wording
<sham1>
> The macros are […];and errno, which expands to a modifiable lvalue
<heat>
oh I think they're saying errno must be a macro but can expand to either "errno" or "*whatever()"
<heat>
so doing extern int errno; #define errno errno is totally valid
<sham1>
That's my understanding
<heat>
yes, i think you're correct, interesting
<sham1>
…and that's horrible
<heat>
i don't see how this changes my life in any way
<sham1>
I personally like the explicit _errno
<sham1>
So I don't need to have the macro be an identity macro
<heat>
errno must be thread-safe for C11 so you pretty much need to expand it into *__errno() or something
<heat>
and you can't use _Thread_local unless you want to break non -std=c11
<sham1>
Or use the _Thread_local which was also added in C11 and does the right thing
<sham1>
Well I don't care
<heat>
:(
<sham1>
That's why I wouldn't implement gets either
<sham1>
Or you can use a macro to switch between the C11 keyword or the GCC/clang/whatever specifier like gcc's thrrad
qubasa has joined #osdev
xenos1984 has joined #osdev
Left_Turn has quit [Remote host closed the connection]
Left_Turn has joined #osdev
Left_Turn has quit [Remote host closed the connection]
Left_Turn has joined #osdev
Left_Turn has quit [Remote host closed the connection]
Left_Turn has joined #osdev
Left_Turn has quit [Ping timeout: 276 seconds]
wootehfoot has joined #osdev
elastic_dog has quit [Ping timeout: 260 seconds]
Left_Turn has joined #osdev
GeDaMo has quit [Quit: That's it, you people have stood in my way long enough! I'm going to clown college!]
elastic_dog has joined #osdev
Left_Turn has quit [Remote host closed the connection]
Left_Turn has joined #osdev
Left_Turn has quit [Ping timeout: 268 seconds]
inegatives has quit [Ping timeout: 255 seconds]
simpl_e has joined #osdev
IRChatter has quit [Quit: ]
<zid`>
ChadGPT: You hiring fuz to add simd to freebsd?
Left_Turn has joined #osdev
Left_Turn has quit [Remote host closed the connection]
Left_Turn has joined #osdev
_xor has joined #osdev
vdamewood has joined #osdev
Turn_Left has joined #osdev
Left_Turn has quit [Ping timeout: 252 seconds]
slidercrank has quit [Ping timeout: 248 seconds]
rnicholl1 has joined #osdev
termin has joined #osdev
[itchyjunk] has joined #osdev
rnicholl1 has quit [Quit: My laptop has gone to sleep.]
dutch has quit [Quit: WeeChat 3.8]
dutch has joined #osdev
bauen1 has quit [Ping timeout: 250 seconds]
Burgundy has quit [Ping timeout: 265 seconds]
termin has quit [Ping timeout: 264 seconds]
bnchs has quit [Read error: Connection reset by peer]
linear_cannon has joined #osdev
vdamewood has quit [Read error: Connection reset by peer]
wootehfoot has quit [Quit: Leaving]
vdamewood has joined #osdev
Matt|home has quit [Quit: Leaving]
<heat>
i'm hiring fuz to add simd to onyx
<heat>
whatever fuz is
<heat>
he/she/they are hired
<heat>
or if its a thing, it's hired
<zid`>
Wow I didn't know you had that much money
<zid`>
you must have like half the gdp of portugal
termin has joined #osdev
Turn_Left has quit [Read error: Connection reset by peer]