<gorgonical>
I'm getting a weird feeling I've somehow broken some cache functionality. A hashtable lookup that's never broken before is exhibiting now the same kind of cache coherence behavior from earlier, requiring a manual dcache flush to show up on the other core
<gorgonical>
I don't like when cache stuff like this breaks because it's very confusing
xvanc has joined #osdev
slidercrank has quit [Ping timeout: 268 seconds]
xvanc has quit [Ping timeout: 255 seconds]
<geist>
yeah
<geist>
you were not doing SMP before and now you are?
<geist>
what changed?
<kazinsal>
ha, yeah, that's something that's pretty universal. "hey this works great" *turns on another core* "what the fuck"
<klange>
I continue to be rather afraid of how little I did in this regard.
<klange>
What mysterious issues lurk in unseen corners?
<Mutabah>
Meanwhile, I reaped the benefits from using rust recently. Turned on SMP, and after getting the bringup logic working - everything Just Worked
<moon-child>
rust won't help you not screw up the cache
<gog>
rust rust rust
<gorgonical>
geist: I don't know if we ever got kitten with multiple cores working on this board, but we've got it working on a different one
<gorgonical>
with multiple cores. that was how we learned about the morass that's inner/outer shareable and how both the tcr registers and the page tables are marked for it separately, etc
<gorgonical>
I don't *think* I did anything to mess with that, but now I'm seriously paranoid lol
<Mutabah>
moon-child: Well, not things like TLB shootdown sure... but other race conditions yep
<moon-child>
kindaaa
<moon-child>
it can at least attempt to draw attention to cases when something is shared
<moon-child>
which you can argue is advantageous
<moon-child>
but you can always manifest your own race conditions at a higher level
<gorgonical>
I haven't changed any of the code to do with memory configuration and stuff, just checked. I did check and the TCR has shareability set to inner, which is the less inclusive choice
<gorgonical>
I should check the page tables themselves now
<moon-child>
not saying useless. But not panacea either
<geist>
yah this may be where you have been missing some memory barriers and now you need them
<geist>
for example updating a page table entry and not DSBing when done
xvanc has joined #osdev
<gorgonical>
Is it possible that this is "IMPLEMENTATION DEFINED" behavior varying? I'm seeing with an alarmingly large amount of the arm cache/translation stuff that the exact behavior is impl. def.
<geist>
not anything in v8
<gorgonical>
I am currently chalking it up to the pine64 board being different from the rk3399. After all, the rk3399 actually has two clusters, meaning in theory there is a difference between inner/outer shareable. But I was under the impression that if PE 0 and 1 are in the same "shareability domain" and you mark pts as inner shareable you shouldn't have to manually manage cahce
<geist>
v8 standardized things pretty solidly. before that there were some exceptions
<geist>
and no. the two clusters should absolutely be inner sharable
<gorgonical>
then what is outer for?? the manual at least suggests clusters could be in different domains
<geist>
otherwise it wouldn't work. all cores that expect to be in the same SMP domain *must* be inner sharable
<geist>
outer is basically deprecated. idea is you could put cores that are off running different things
<geist>
or say, a GPU or whatnot
<geist>
but i think in practice nothing uses it
<gorgonical>
oooh a gpu is not a bad idea for is
<gorgonical>
but I see what you mean
<gorgonical>
but basically it means that for all systems if you mark it as *either* shareability then it should work as you expect, without explicit cache management?
<geist>
well, not entirely sure what you're asking there
xvanc has quit [Ping timeout: 255 seconds]
<gorgonical>
What I'm really getting at is that since inner is fully covered by outer and outer is deprecated, it really means you have no-share and inner/outer
<geist>
well, and 'sys' which is the outer to outer
<geist>
at least in things like DSB and whatnot
<geist>
think of it this way: inner sharable is for synchronizing between cpus in an SMP system
<geist>
the SYS domain is for synchronzing with things like devices
<geist>
so if you just want to say insert a barrier so that another cpu sees it: dsb ish
<geist>
but if you want it to do the whole thing it's 'dsb sy'
<gorgonical>
Holy crap those acronyms just clicked
<geist>
in linu for example a bunch of these are hidden behing 'mb()' macros
<gorgonical>
Before they just looked like perl sigils to me but it's "data sync barrier inner-share"
<geist>
like smp_mb vs mb. and in the case of smp_ it'll probably be ish
<geist>
yep!
outfox_ has quit [Ping timeout: 255 seconds]
<gorgonical>
Hmm. Okay maybe I'm forgetting some basic CS theory here but what's the point of setting them shareable if, to be sure it happens, I have to dsb ish after each malloc for a smp-shared resource?
<geist>
because ARM is a weakly ordered architecture (vs x86)
<gorgonical>
Like, is the setting in tcr_el1 and the pt entries just a hint to the caching engine about what to flush and stuff when I *do* run dsb ish?
<geist>
without actual barriers at particular points, other cores are not guaranteed to see writes in any particular order
<geist>
it's still cache coherent, but the order at which things happens is undefined
<gorgonical>
Gross. I'm also shocked this wasn't somehow a problem before
<geist>
so you use barriers like dmb/dsb/isb at various points or use isntructions with built in barriers (lots of atomics do) so that it's not a problem
<gorgonical>
I mean it's not shocking considering this is originally an x86 OS but we *did* port it
<geist>
in general if yuo aren't doing a lot of volatile sharing, and you're probably using locks and atomics, it just works
<geist>
since locks and atomics almost always have a barrier built in them already
<gorgonical>
oh that's a good point
<gorgonical>
what's the acronym for dmb then?
<moon-child>
data memory barrier?
<geist>
ie you grab the mutex (barrier), now you can fiddle with the object(s) all you want, and writes to the cpu can happen in any order, then you go to release the mutex (barrier), now all other cpus see everyting occurred
<geist>
(this is where acquire and release semantics come into play, where the barryer can operate in either direction or both
<geist>
)
<gorgonical>
ah
<geist>
and yeah DMB is a weaker and more subtle form of DSB
<geist>
DMB just informs the cpu that when it gets around to writing out stuff it has to do it in this particular order (before + after)
<gorgonical>
Then in theory if this stuff was working before I need to go snoop around my atomics/locks and make sure I didn't break something there. We use locks and such properly
<geist>
but doesn't otherwise stop the world
<geist>
DSB is more of a stop the world, make it get out into the cache subsystem
baine has quit [Ping timeout: 248 seconds]
<geist>
i think you'll also notice the cache flush routines have a DSB at the end right?
<gorgonical>
yes
<geist>
same with TLB sync routines on arm64
<geist>
that DSB also acts as a 'make sure those pending cache flushes/tlb syncs happened'
<gorgonical>
yeah it's transactional, right
<geist>
since they are *also* weakly ordered
<gorgonical>
It is more flexible but also more complex
<geist>
think of both of those kinda operations are a special kinda bus write, essentially. queued up with all the other writes the cpu may be doing
<geist>
yah makes you appreciate (or be grossed out) by all the efforts x86 cpus myust go through to make after appear in order
<geist>
even if they aren't really
<gorgonical>
seymour cray was right
<gorgonical>
multicore is a mistake
<gorgonical>
lol
baine has joined #osdev
<geist>
x86 cpus are tracking a fairly substantial amount of state to keep everything looking in order from the outside
<moon-child>
from what I understand from what david chisnall said, the effect of tso is basically just that you have to have a bigger write buffer
<moon-child>
weak ordering def better though
<geist>
right you have a large write buffer and strict dependencies between entries in it
<moon-child>
mehh
<moon-child>
need lots of fancy dependency tracking anyway
<moon-child>
I actually kinda wonder what would happen if you moved more of that out of the uarch into the arch
<geist>
true, though it's much simpler to just slam stuff out as the cache lines become available.
<geist>
but it's a good question: what is the real cost of all of that
<moon-child>
eg don't do memory disambiguation in hardware; instead, have alias tags on loads/stores
<geist>
really the apple M1 cpus are probably the best way to test that theory, since they support both modes dynamically
<moon-child>
maybe. Maybe they underestimate the effect, since you're still paying in area
nyah has quit [Quit: leaving]
<geist>
they n eed it because of x86 emulation, on top of an otherwise weakly ordered machine
<geist>
but in that case yeah they have to build the silicon to do it, but can turn it off for regular ARM code
<geist>
so presumably there must be a win, since they have both situations at the same time
<geist>
otherwise they could just choose to leave it on all the time since they've already got the silicon for it
outfox has joined #osdev
<moon-child>
yeah, but point is there might be an even bigger win if you could take that silicon and use it for something else
<gorgonical>
so do the ldaxr/stxr instructions do something like dmb/dsb? They don't, right? This is just for synchronizing on the lock address itself
<geist>
moon-child: oh 100%. they dont have that choice but most arm cores of course do, and thus dont have the overhead of silicon for it
<moon-child>
there are some other big arm cores with a tso mode iirc
<geist>
gorgonical: they do, the 'a' is the acquire part
<geist>
otherwise it's just ldxr
<geist>
and similarly stlxr vs stxr (the 'l' is for release)
<gorgonical>
That's disturbing then
<geist>
so using one or the other n either end gets you an acquire/release or both (seqcst)
<geist>
if you put both then you have created a full before & after barrier
<geist>
otherwise an acquire or relase barrier only works in one direction, which is fine for the nomenclature: you're acquiring access to the data vs releasing it
<gorgonical>
Then I must be misunderstanding something
gog has quit [Ping timeout: 255 seconds]
<gorgonical>
Or something is super borked
<gorgonical>
I allocated this htable inside a spinlock, which has ldaxr for locking and stlr for unlocking the lock
<gorgonical>
But if I don't explicitly flush_dcache_area on the *pointer* for the htable, the pointer shows up in core 1 as null.
<geist>
did you enable the cache on core 1?
<gorgonical>
But the malloc is inside the spinlocks
<geist>
what kinda cores are these?
<gorgonical>
These are the standard cores on the rk3399, so a72
<gorgonical>
Cache should be enabled on core 1
<geist>
double check that
<geist>
(almost certainly is because otherwise your spinlocks would asplode)
<geist>
are you positive your spinlock works properly?
<gorgonical>
Positive in the sense that they have been working for years at this point and to port it about 18 months ago all we did was copy Linux's spinlock primitives for the arch port
<gorgonical>
We did not re-engineer the asm for the spinlocks
<geist>
when you initialize the second core did you invalidate it's cache? (should be fine, but a good idea)
<geist>
in case it has stale entries with garbage in it
<gorgonical>
Yeah part of head.S has a big dcache flush I believe
<geist>
flush or invalidate?
<geist>
ie, clean vs invalidate vs clean + invalidate?
<geist>
(to use arm's nomenclature which is very precise)
<gorgonical>
bl __flush_dcache_all; ic iallu; dsb sy
<geist>
on the secondary core?
<gorgonical>
Yes
<geist>
before the cache is enabled?
<geist>
what precisely dose that flush dcache routine do?
<gorgonical>
Should be. We go from cpu_setup to enable_mmu
<bslsk05>
github.com: kitten/cache.S at master · HobbesOSR/kitten · GitHub
<geist>
yeah that's probably fine. double so if you boot it from PSCI which i think guarantees that the cache is clean
<gorgonical>
I wasn't responsible for this code so I don't know it well
<geist>
hmm, yeah that code is not really a great idea, but it's a super hard problem to generically do
<geist>
so for example that code uses a `dc cisw` at the bottom of it
<geist>
so it's really clean+invalidate
<geist>
which strictly speaking isnt a good idea here because that it does is it causes the secondary cpu to write out any stale cache lines it may have
<geist>
since the cache isn't enabled, and/or it was initialized with garbage it could hypothetically overwrite something
<geist>
but it probably doesnt
<geist>
usually the right idea is if you really want all your ducks in a row either dont do anything (because PSCI guarantees cpus come up with an empty cache) or do a pure *invalidate* over your cache
<geist>
but *only* to the point of unification
<geist>
so you end up needing like 8 variants of that
<geist>
i kinda doubt ay of this is a problem
<gorgonical>
I have a suspicion that running in secure mode is contributing to the problem but I am not sure
<gorgonical>
So we flush+invalidate, then load sctlr_el1 val which enables caching. Since it's the same kernel code the pts and control registers are the same, so they should all have the same sharing settings
<geist>
yeah
<geist>
must be something you dont know could be a problem, and thus aren't reporting
<geist>
ie, everything you have told me about seems correct at least on the surface
tiggster79 has joined #osdev
xvanc has joined #osdev
heat has joined #osdev
<heat>
hell
<heat>
i saw i missed a RUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUST moment, so there it is
xvanc has quit [Ping timeout: 248 seconds]
xvanc has joined #osdev
frkzoid has joined #osdev
wand has quit [Remote host closed the connection]
gxt__ has quit [Remote host closed the connection]
gxt__ has joined #osdev
wand has joined #osdev
heat has quit [Read error: Connection reset by peer]
heat has joined #osdev
[itchyjunk] has quit [Remote host closed the connection]
xvanc has quit [Remote host closed the connection]
Vercas has quit [Remote host closed the connection]
Vercas has joined #osdev
<geist>
got the visionfive 2 rv64 running
gxt__ has quit [Remote host closed the connection]
<geist>
better performance than i expected. sits nicely alongside quad a53 boards i've used
<geist>
which is exactly in line with the kinda cpu tech thats in it (quad sifive u74 cores @ 1.5Ghz)
gxt__ has joined #osdev
Arthuria has quit [Remote host closed the connection]
bgs has joined #osdev
<gorgonical>
Here's a thought. If I run core 0, which should be releasing the spinlock, in a loop waiting for core 1 to change some sentinel, if I run dsb sy each loop iteration will that sort of act as a total ordering flush for the system?
<gorgonical>
Just for debugging purposes?
<gorgonical>
Core 0 reads that the spinlock is released but core 1 doesn't and is stuck forever
slidercrank has joined #osdev
xvanc has quit [Remote host closed the connection]
<qxz2>
what are your opinions of go? i know it's mostly meant for distributed/cloud apps.
<zid>
oly thing I really know is that the binaries are absolutely enourmous and the ecosystem is so fragile that people just maintain all their own deps
<qxz2>
i think they're statically linked
<qxz2>
which probably explains the bloated binaries
<zid>
some language feature like reflection or debug api means they ahve to include the source for every line and stuff in everything
<zid>
there's still afaik an open bug for "yea, maybe hello world shouldn't be 10MB?"
<qxz2>
hah
<zid>
that former thing is also annoying to corps like nvida who use it afaik
<zid>
I can't remember what their solution was, source obfuscation or something
<zid>
or hacking the binaries up
<qxz2>
nvidia is known for using go? i didn't know that
<zid>
no, they're known for making graphics cards, you'd think you'd have heard of them before :P
<geist>
i suspect go is quite good at doing precisely what it sets out to do and isn't so great in other contexts
<geist>
ie, is more domain specific than it probably wants to be
<qxz2>
i'm very familiar with nvidia
<qxz2>
what do they use go for?
<geist>
but i can't say i have a tremendous amount of go experience
<geist>
i was briefly swept up in the hype 10 years ago or so
<qxz2>
the generics are idiosyncratic
<qxz2>
it's an interesting reaction to typical OO langs
<qxz2>
composition instead of inheritance
<qxz2>
interface types
<geist>
gorgonical: how are you configuring the page tables?
<geist>
there's a shared/no shared bit
<zid>
qxz2: idk, wouldn't surprise me if some random stuff like the debugger or experience etc was written in it, I was just talking to someone working on some go at nvidia listening to his rambling
<qxz2>
i googled around and saw nvidia job ads looking for go experience, mostly for backend type dev, which makes sense
<geist>
and/or web backend where i think it shines
<qxz2>
distributed systems are go's niche
<qxz2>
"Building automation for routine maintenance tasks for GPU farm
<qxz2>
fun
<geist>
yah makes sense.
<qxz2>
concurrency is built in. i think the runtime has its own scheduler.
<zid>
neat, there's a tool that can unpack the nvidia driver packages, do the installation by hand with better configurability
<zid>
and apply a driver patch to enable MSI
Vercas has quit [Quit: Ping timeout (120 seconds)]
Vercas has joined #osdev
<gorgonical>
geist: the shared/unshared bit refers to the two bits in the pte right?
<gorgonical>
where 11 is ish and 10 is osh?
<gorgonical>
or so
<geist>
iirc there's a NS bit
<geist>
no. it's another one
<gorgonical>
That's the non-secure bit
<geist>
yeah okay, maybe there's another one
<geist>
basically it says 'dont bpther doing cache coherency with this' i think. basically unused. lemme see
<qxz2>
is this x86_64?
<immibis>
ARM documentation tells me that cache coherency is system-specific, but that is from the point of view of the single-core CPU
<geist>
ah yeah i guess it is bit 8
<qxz2>
oh nm
<gorgonical>
Because I'm losing my mind over here. I can't even get the spinlock to communicate even when I loop dsb sy
<bslsk05>
github.com: lk/mmu.h at master · littlekernel/lk · GitHub
<gorgonical>
qxz2: yeah arm64
<geist>
anyway you def want to set inner sharable there on all the PTEs
<gorgonical>
Yeah those are set
<geist>
yah using non sharable pages there is a feature that i dont think anything uses, and is possibly unimplemented in most cores
<geist>
but still worth setting
<gorgonical>
it is definitely implemented in the pine64
<gorgonical>
lol
<geist>
there's some verbiage on a lot of this inner/outer stuff that an implementation may actually implement something 'larger'
<gorgonical>
That was how we discovered it was a feature in the first place
<geist>
ah
<gorgonical>
When we first ported kitten to arm64 fully
<geist>
so just to be clear this is the very first time you've tried to enable SMP on ARm64?
<geist>
whereas you have working SMP on x86?
<gorgonical>
No, this is the first time I've tried smp on arm64 in secure mode on the rockpro64
<gorgonical>
We have smp working in regular mode on qemu and the pine64
<geist>
hmm
<gorgonical>
yes exactly
<geist>
can't say i think there's much of a difference there, except the whole secure bit in the page tables
<gorgonical>
And these are basically the exact same problems that we solved the last time by fixing coherency issues
<geist>
maybe that interacts with the cache subsystem in a weird way
epony has quit [Quit: QUIT]
<gorgonical>
That's my only thought but I did look at the manual with some scrutiny and ctrl+f and didn't find any... clear indication that it might mess with it
<geist>
but in general secure vs insecure mode should be hidden. the cpu even boots in secure mode, so it's possible that a random linux is actually running in secure mode, just because there's nothing that took it away
<geist>
probably does a lot of the time anyway
<gorgonical>
yes that's right. your bootloader actually needs to be sure and untick the box, but if you leave it on most stuff will 'just work'
<geist>
and the secure/nonsecure bit only really kicks in when you have both modes actie at the same time, otherwise they dont really *do* anything
<gorgonical>
yes
<gorgonical>
right
<gorgonical>
so in theory I wouldn't expect the secure bit to have such dramatic impacts on the coherence subsystem
<gorgonical>
Unless something horriyfing like one of the cores running in secure mode and the other one not is happening
<gorgonical>
WAit
<geist>
yah there's some complicated verbiage about generating cache entries in one mode and then trying to use it in another, etc
<geist>
oooh.
<gorgonical>
Boy I hope that's not true lol
<geist>
that might do it, because i think secure cache lines can't really be accssed from non secure cpus, etc
<gorgonical>
trusted firmware and the psci interface sucks anyway man. It's always something
<geist>
and then i honestly dont know what happens, up to and including UNDEFINED
<gorgonical>
trusted firmware doesn't even allow secure world callers to issue psci calls
<geist>
having fun building linux here on this riscv machine. it's slow! but it's cortex-a53 slow, so really that means it's performing as expected
<gorgonical>
I'm jealous. I haven't had time to play with my riscv machines for a while and won't for more time
<gorgonical>
dissertation research and stuff looms
<gorgonical>
But I badly want to port kitten to the d2 riscv core I have
<geist>
yah sadly nothing on the market yet supports virtualization mode, except qemu
<geist>
looks like some of the new sifive performance cores, the P670 in particular, does
<geist>
but dont know when that'll show up in hardware
<gorgonical>
I am very pleased that sifive has so much traction. I was working somewhere we could afford the boards but I'm happy someone's actually doing it
<gorgonical>
we had the unmatched
<geist>
yah i have an unleashed around here somewhere
<geist>
and an unmatched
<gorgonical>
The d2 I got is this goofy clockwork pi devterm thing
<gorgonical>
It comes with a thermal ink receipt printer
<gorgonical>
...
<geist>
probably will get ahodl of the one of the horse creeks when they come out, but they're going to be P550 based, which is just before the cut to get virt, etc
<gorgonical>
This nonsecure core theory can be tested, too. I haven't actually put the code in kitten to turn on the trustzone memory controller for the kernel image
<gorgonical>
That would quickly tell me if secondary core is secure or not
<gorgonical>
geist have you played with zig at all? It came to my attention recently and its constant presence on HN makes me wonder about it
xvanc has joined #osdev
xvanc has quit [Remote host closed the connection]
<geist>
hmm, zig the language no. someone mentioned it at work as something to look at
<geist>
and they seemed to be pretty impressed
xvanc has joined #osdev
<gorgonical>
I haven't done much work in it but it seemed very simple and even pleasing. In the way that c++ is impressive but because of its complexity, zig felt impressive because of what it seems to be able to do while still looking like c to me
<gorgonical>
I wrote a little stub kernel in D's betterC subset and I remember wishing "why isn't this just a whole language"
<moon-child>
isn't it a whole language?
<moon-child>
also: betterc is mostly a marketing thing
<zid>
am I allowed to pronounce that like "beturk"
<gorgonical>
nobody can stop you, zid
<zid>
To make something turkish
<gorgonical>
moon-child: how do you mean about it being marketing?
<gorgonical>
that it has neat interop or something like that?
<moon-child>
there's not much point in using it over 'regular d'
<gorgonical>
I mean unless you're trying to do something like a simple kernel
<zid>
using regular D also lets you make crappy memes about sex
<moon-child>
gorgonical: no
<moon-child>
omg
<moon-child>
'use lldb', they said
GeDaMo has joined #osdev
<moon-child>
'it's great', they said
<moon-child>
I tried it, and it span up to 100% cpu doing who knows what
<zid>
First they came for the lldb users but I did not speak out, for I was sane
<moon-child>
never doing that again
* moon-child
remains annoyed gdb doesn't work on mac
<gorgonical>
I've been using gnu global and ggtags recently and I quite like it
smach has joined #osdev
epony has joined #osdev
levitating has quit [Ping timeout: 268 seconds]
levitating has joined #osdev
Vercas has quit [Quit: Ping timeout (120 seconds)]
Vercas has joined #osdev
gog has joined #osdev
<gorgonical>
Good job forgetting for nearly an hour that ioremap_nocache exists and fumbling around like a dingus with head.S
<gorgonical>
Probably time to sleep for the night
danilogondolfo has joined #osdev
nyah has joined #osdev
Turn_Left has joined #osdev
Left_Turn has quit [Ping timeout: 264 seconds]
danilogondolfo has quit [Remote host closed the connection]
<geist>
hmm i really should give zig a shot though
<geist>
that's at least two people that have recommended it for all the reasons i'm interested in
levitating has quit [Ping timeout: 255 seconds]
danilogondolfo has joined #osdev
levitating has joined #osdev
Vercas has quit [Remote host closed the connection]
Vercas has joined #osdev
gildasio has joined #osdev
tepperson has quit [Ping timeout: 248 seconds]
Vercas has quit [Remote host closed the connection]
Vercas has joined #osdev
Amorphia is now known as Rust
epony has quit [Remote host closed the connection]
zxrom has quit [Quit: Leaving]
epony has joined #osdev
Rust is now known as Amorphia
Vercas1 has joined #osdev
<epony>
is it supported by GCC ?
Vercas has quit [Remote host closed the connection]
Vercas1 is now known as Vercas
<mjg>
geist: zig?
<mjg>
geist: quite frankly i would just spend time on rust man :)
<mjg>
geist: why the zig
<davros1>
geist have you tried any other C/C++ alternatives cough rust cough
<davros1>
I must admit although 100% capable rust is a PITA for low level (IMO).
<mjg>
excuse me sir, can i talk to you about a fast and memory-safe alternative to whatever you are using right onw?
<davros1>
Look what I did there with a disclaimer before I get onto its virtues
<davros1>
Seems there are some other options appearing that offer better C++ interoperability
<davros1>
I can't afford another language switch though
<pitust>
isnt zig the language where the compiler is a minefield?
<mjg>
is it?
<mjg>
i think that's the onme which compiles to c
<davros1>
They have their own backends
<mjg>
can't be arsed to check
<davros1>
Its not compile to C, they do LLVM and their own codegen
<davros1>
The aspect in which the compiler may be a minefield perhaps is it lets you run code at compile time
<davros1>
To be fair thats probably less of a minefield than an accidentally discovered Turing complete language in the type system
<pitust>
afaik the zig codegen itself can be buggy
<davros1>
I must admit I thought they were biting off more than they could chew even trying that
<davros1>
Rust has stuck to LLVM
<pitust>
even with llvm
<davros1>
I even wish rust could compile to C
<davros1>
Heh ok
<pitust>
you can compile llvm to c with llvm-cbe
<davros1>
Well maybe if they weren't trying to write their own codeine they could fix those bugs
<davros1>
Oh I thought that had all bitrotted (last time I tried)
<davros1>
Rust -> C would have made getting onto some platforms a little easier
<mrvn>
why oh why does /proc/pid/fd/ have a link count of 2? Should be 2+<num fds> :(
<sham1>
OpenGL should be able to tell you why the operation fails
<sham1>
I just don't remember the API off the top of my head
tepperson has joined #osdev
<tepperson>
this is a dump from qemu with gdb, what might cause this behavior? I'm seeing instructions executed that don't exist. https://pastebin.com/6fuJwtwB
<bslsk05>
pastebin.com: weird long mode behavior? - Pastebin.com
gxt__ has quit [Remote host closed the connection]
<gorgonical>
turns out the secondary core isn't starting in secure more, because life is suffering and nothing is simple
<mjg>
gog: you are not as funny as solaris diaspora
<gog>
who
<heat>
mjg, you know what's worse?
<mjg>
OXIDE
<mjg>
an actual company using rust
<gog>
o
<heat>
BSD people that have bsd in their nickname/email
<mjg>
i mean RUST
<mjg>
dude there is a not-bsd guy who literally has a nickanme ending in bsd
<bnchs>
rust has a really bad syntax
<zid>
OXIDE is a bad language and also a bad guess in wordle
<zid>
double whammy
<mjg>
bnchs: that is a widespread opinion
<mjg>
which i do share :)
<mrvn>
What CPUs support 512GB pages? Even 1GB is optional.
<bnchs>
like i have to type more symbols in rust than C
<gog>
mrvn: none of them :P
<gog>
maybe some hypothetical risc-v machine
<heat>
mjg, there's this guy in linux kernel dev that named himself after you
<mrvn>
What you can do is a fractal mapping. Then you can map a lot of memory with a single 4k page.
<heat>
and this "linus" guy that named himself after linux
<bnchs>
heat: is he the guy who shills microsoft pluton?
<mjg>
heat: did you know original name for the kernel was FREAKS or something like that
<heat>
mjg, yep
<heat>
bnchs, mjg does not shill for pluton
<FireFly>
linux torvalds
<gog>
he made linux os
<mjg>
it's unix, i know this
<gog>
unix is really good
<gog>
i know it
<heat>
gog, how much do you like fork() and how little do you not like CreateProcess
<mjg>
nah dawg fuckitix is the feature
<gog>
creating processes is cringe no matter how you do it
<gog>
i will never
<mjg>
sounds like templeos is for you then
<heat>
do you only create threads?
<mjg>
no processes
<mjg>
i think
<gog>
no i got rid of that code
<gog>
single process forever
<mjg>
you basically print 'lol' in a tight loop?
<gog>
yes
<mjg>
aight i would sign off on that kernel
<heat>
depends
<mjg>
i would accept LoL
<gog>
it uses PrintLolFactory to do it
<heat>
mjg would only sign off on it if it was OPTIMAL
<mjg>
right
<heat>
PESSIMAL CODE = PESSIMAL
<mjg>
no printf("%s\n", "lol");
<mjg>
o rsimilar\
<heat>
i hope your locking primitives are on point
<mjg>
in fact i would proably only be satisfied if you hacked the cpu
<mjg>
and patched the microcode to do it the fastest way it can
<mjg>
which is probably inaccessible thorugh regular code
<mjg>
you could even possibly block SMIs
<mjg>
for more performance
<mjg>
so ye, ultimately NACK, try harder
<heat>
not-ok mjg@
<mjg>
what he said
<kof123>
i watched that temple vid linked the other day...the funny part was ba ha ha ha ha ha ha
<kof123>
i never used it because 64-bit only
<mjg>
you are somehow stuch with a 32-bit cpu?
<mjg>
*today*?
<kof123>
no, just saying why i never used it
<heat>
he only enjoys Portable(tm) Software(tm)
<bnchs>
mjg: what about a CPU that does operations on 64-bit integers, but has a 32-bit address bus
<kof123>
i was hoping his typing "notes" he would show some move like: "The Oberon System has an unconventional visual text user interface (TUI)" and plan 9 IIRC
<mjg>
bnchs: on your laptop?
<mjg>
i got a product on a 32 bit arm *today*, the pain is real
<gorgonical>
bnchs: in my head your handle is "bean cheese"
<heat>
gorgonical: in my head your handle is "gorgonical"
<kof123>
it is true i would never go for his U8 U16 U32 U64
<kof123>
i have worse ideas than he does i think, so i show a bit of respect
<mrvn>
Poll: What are "and *plant shaped C4 charges* on the four pipelines". Explosives shaped like plants so they aren't so obvious, right? Thank you google translate.
<gorgonical>
Sounds like you mean you are placing shaped charges to achieve a more targeted effect
<gorgonical>
the phrase is garden path-y though
<mrvn>
gorgonical: obviously. "plant shaped charges" not "plant-shaped charges". But google doesn't get that.
<gorgonical>
Ooh I see what you're saying
<mrvn>
They reported plant-shaped charges on the NEWs on the german TV channel.
<bslsk05>
yosefk.com: C++ FQA Lite: Input/output via <iostream> and <cstdio>
<tepperson_>
ok i now have the first 2 megabytes of ram mapped with my page tables (with a 2mb page directory entry). jmp 0x104080 actually jumps to 0x407e, what might cause that?
<sham1>
Just use std::printf like a sane persohnj
<zid>
user error?
<zid>
either the assembler, or one of the debugger tools got told/asked the wrong thing somewhere
<mjg>
lol > or some reason, it doesn't throw an exception (it's not really bad, because what's really bad is C++ exceptions).
<davros1>
What I'm seeing for pixel art there is quite different to the tailored palette work of classic pixel art.. but the Dalle things above were closer.
<geist>
will be interesting to see when the first game that somehow integrates this sort of image generation in the game itself
<davros1>
Still it doesn't matter. I think its still a good "quality : effort" tradeoff.. doesn't have to be perfect
<geist>
cant think of how that'd owkr, but you could imagine some sort of clever puzzle thing where part of the puzzle is generated art
<geist>
or machine generated art treated as a level, or whatnot
<davros1>
Heh yeah Geist - stable diffusion on the highest end GPU can spit out new images in a couple of seconds . Imagine a scroll speed tuned such that you're always moving in to newly generated things
<geist>
yah. and of course that'll get moare optimized over time, etc
<davros1>
But even without goign that far.. being able to make random mazes or whatever , and just visually enhance them with "Img2img" would be great
<geist>
yah, probaby some sort of indie game tries it first, with it being the main trick of the game, and then slowly becmes more standard
<davros1>
I think for AAA hand tuned art will win in 3d. Generative will produce a rough dreamlike style that will suit indies (quality:budget)
<davros1>
Indies and modders
<geist>
yah
<geist>
i was planning on piddling with stable diffusion maybe this weekend. my older 1080ti should still be able to crunch numbers fairly well
<davros1>
Things like Minecraft/roblox ..
<GeDaMo>
It might be possible to make smaller models trained for specific games / styles
<davros1>
Yeah that should do 10-15 seconds per image.. better than hugging face certainly
<geist>
iirc the 1080ti is somewhere like a 3060 iirc
<GeDaMo>
Facebook / Meta just released a paper on their new LLM which showed a smaller model trained with more data can match GPT-3
<geist>
assuming it doens't need specific hardware that the new rtxes have
<davros1>
Its worth grabbing a 3000 series card or even 4000 if you can
<geist>
yah i've been putting it off unti i can nicey find a strong replacement for it
<geist>
but this may be something that pushes me to
<davros1>
I'm fine with splashing out on a big GPU but electricity is the limiting factor really
<davros1>
UK electric bills are high
<davros1>
If it wasn't for that I'd be running 2
<geist>
yah i get ya there
<davros1>
SD img2img has also encouraged me to get back to amateur art
<davros1>
Doing my own doodles and AI enhancing
<davros1>
AI can't do 3d game ready yet, hence the interest in retro styles
<davros1>
And hence the interest in retro hardware again hah
<bnchs>
AI isn't accurate yet lol
<gog>
it's not very intelligent
<gog>
like me
<davros1>
Neither are people in many ways,although I know what you're saying
<bnchs>
yeah, most of the times you catch it saying dumb shit
<davros1>
chatGPT bullshits alot, like a human ..
<gog>
yes
<bnchs>
especially when you tell it to explain something
<gog>
chatGatekeep, chatGaslight, chatGPT
mctpyt has quit [Ping timeout: 252 seconds]
<davros1>
But the art generators - if you use 'img2img' you can sketch, say what it's supposed to be, and it'll detail for you. This is pretty good quality/control/productivity balance. man+machine
<tepperson_>
i asked chatgpt to make a dummy kernel driver and it used structs that don't exist in any kernel
<davros1>
Yeah I wont use it for code
<gog>
it doesn't really have a capacity for abstract reasoning
<gog>
such a thing can't really code
<davros1>
Script kiddies are getting excited by it though..
<bnchs>
yeah
<bnchs>
"OH NO CHATGPT CAN MAKE RESPONSES, OUR PROGRAMMING JOBS WILL BE DOOOOMED!!!"
<bnchs>
can't even make a working kernel driver
wootehfoot has quit [Read error: Connection reset by peer]
<mats1>
its not good but its not useless
<mats1>
just needs some care and sanitisation
<mats1>
like cvs where there's six self checkout stations and one guy to tend to them / greet guests
<sbalmos>
those six checkout stations require 6 truckloads of receipt printer paper
<mats1>
there's great business value in eating the low hanging fruit jerbs where retards are googling and including left pad libraries
<bslsk05>
ke0z/VulChatGPT - Use IDA PRO HexRays decompiler with OpenAI(ChatGPT) to find possible vulnerabilities in binaries (17 forks/203 stargazers)
GeDaMo has quit [Quit: That's it, you people have stood in my way long enough! I'm going to clown college!]
gabi-250_ has quit [Quit: WeeChat 3.0]
gabi-250_ has joined #osdev
<heat>
geist, do you have any perf numbers on 5-level paging vs 4-level paging?
<geist>
i do not. on x86?
<heat>
it sounds like something you'd have
<heat>
yeah
<heat>
also cc mjg
<geist>
no i haven't actually fiddled with any machines with 5 level enabled
<geist>
hypothetically with page table caching and aggressive data caching there's probably not *too* much of a hit
<geist>
possibly the bigger hit is simply that it allows the system to be even more scattered
<mjg>
i don't, but i would expect intel to have a hard-to-find paper with it
<mjg>
alternatively lkml with submiion for la5
<geist>
FWIW riscv has thesame thing now too, but surprisingly linux mainline seems to only currently support sv39
<gorgonical>
geist: isn't that because no hardware really supports anything higher than sv39?
<heat>
i think linux should enable PML5 by default?
<heat>
huh you sure it only supports 39?
<heat>
i think they usually lock that stuff behind a CONFIG_
heat has quit [Remote host closed the connection]
<geist>
yah, except qemu and probably stuff in dev
<gorgonical>
I can't help but wonder if the kernel support is just lagging along then
heat has joined #osdev
<geist>
or it's sitting in a branch yet and hasn't been merged
<geist>
heat: yeah it seems to only have the SV39 config option
<geist>
obviously someone will add it, just don't see it yet
<heat>
at the very least they have layouts for 48 and 57
<geist>
yah makes sense
<geist>
may as well design it even if it's not a build option yet
<netbsduser>
on chatgpt: it has declined significantly, in november it was able to give me competent advice and spot real issues, now it can barely match a typical stackoverflow answer and it only spots superficial issues. i was really disappointed to see how much it declined
<sham1>
As an OpenAI language model, I cannot e expected to actually be useful at this time.
<heat>
geist, yeah you're right, seems to assume 39 only
<heat>
very weird IMO
<tepperson_>
i thought i would try bochs, but on ubuntu it can't even load grub2
<heat>
48 sounds like the no-brainer (because of x86, etc)
nj0rd_ has quit [Ping timeout: 255 seconds]
<heat>
i guess they wanted to support smaller devices?
<heat>
*shrug*
epony has joined #osdev
<heat>
does android still do 39-bits on arm64 too?
<geist>
well, yeah i think all the existing hardware except for qemu only supports up to sv39. the next batch will probably pick up sv48, but it does seem that 39 is the general sweet spot for current non datacenter hardware
<heat>
i have not messed with support multiple va sizes yet
<heat>
supporting*
<geist>
so an annoying thing i just discovered about my vf2 board
<geist>
it seems to not support that many ASID bits
<geist>
possibly 0, which is valid. linux dmesg shows that it basically decides it can't do ASID because not enough bits
<geist>
and the logic in linux to determine that is (num_asids >= 2*NR_CPUS)
<geist>
since NR_CPUS is 8 in this particular kernel, that means it must have declared less than 4 bits of ASID support
<geist>
(probably 0)
m5zs7k has quit [Ping timeout: 264 seconds]
m5zs7k has joined #osdev
Turn_Left has joined #osdev
Left_Turn has quit [Ping timeout: 252 seconds]
valshaped has quit [Quit: Gone]
valshaped has joined #osdev
SpikeHeron has quit [Quit: WeeChat 3.8]
heat has quit [Read error: Connection reset by peer]
heat has joined #osdev
Vercas has quit [Quit: Ping timeout (120 seconds)]
Vercas has joined #osdev
SpikeHeron has joined #osdev
<mjg>
yo
<mjg>
what, if any, cpus today do *software* tlb?
<mjg>
not mips
<mjg>
something which is not slow
<mjg>
:]
<\Test_User>
> not slow
<\Test_User>
> run code for every (uncached) memory access to translate pages
<\Test_User>
seems to conflict a bit to me
<heat>
mjg, IA64 you FUCKEN BITCH
heat has quit [Remote host closed the connection]
heat has joined #osdev
mctpyt has joined #osdev
mctpyt has quit [Ping timeout: 252 seconds]
moberg has joined #osdev
<heat>
geist, i'm starting to think that riscv really is digging its own grave with i n f i n i t e e x t e n s i o n s
<heat>
in this case, infinite optional crap
bradd has joined #osdev
robem has joined #osdev
thaumavorio_ has quit [Quit: ZNC 1.8.2 - https://znc.in]
<geist>
there appears to be an effort to standardize via the riscv profiles stuff
thaumavorio has joined #osdev
<geist>
ie RVA20, RVA22, etc
robem has quit [Changing host]
robem has joined #osdev
robem has left #osdev [#osdev]
<geist>
would be somewhat akin to armv8.x in that you define a series of increasing profiles that include previous ones and add to, with a list of mandatory and optional bits
<chibill>
Honestly I start writing an OS, get to where I can print to the screen (So basically nothing) and then just sort of can't decide how to proceed because my whole drive to write an OS is to just do it for fun so I have no real goal. :(
<geist>
well, you can make interrupts work. that's fun
<heat>
for fun != real goal
<heat>
everyone here does this for fun
<heat>
no one's getting rich off this
<heat>
erm
<heat>
for fun = real goal
<chibill>
Like even for fun I have no end goal of where I want to end up xD
<geist>
right i find it fun to just climb up the tech tree
<geist>
ie, start adding more and more features akin to a Real OS
<geist>
that's fine. treat it as a journey
<geist>
a series of features you add
robem has joined #osdev
<heat>
chibill, what features do you like from your system (linux, etc)?
<heat>
*that do not involve graphics*
<heat>
think small, and use that as an objective
<heat>
like "oh I really like this bash thing to screw around", so you work towards that
<heat>
or an http server, etc
<chibill>
Hm. Just realized I can set a goal of at least having as much stuff as XV6 (So a basic terminal system and file system access of some sort.) Biggest challenge for me is I am working in Rust. I feel like I am going to run into issues when I need access to thing in multiple places. Might start over in C.
<mjg>
oh noez
<mjg>
someone had a kernel written in rust
<mjg>
you could probably use it as a reference when stuck
<heat>
note that xv6 sucks
<heat>
(IMO)
<chibill>
mjg: You mean the RustOS blog thing?
<heat>
xv6 is a stupid simple example of a unix-ish kernel but man the code is not really high quality
<heat>
much like the original UNIX I guess
<mjg>
chibill: no, made by a local
<geist>
but that's a goal, do a better job than xv6
<mjg>
Mutabah: where ya kernel at
* Mutabah
is away (Sleep)
<heat>
kernel is sleep
<mjg>
dafaq that mirc shieeet
<chibill>
heat: I agree, had to do some deep inside work on it for a College class. (Adding new syscalls, file systems blocks, a way to show the memory layout and things like that.)
<heat>
what's really irking me is that they use __sync_synchronize but then never bother to use any of the other ones
<mjg>
heat: it is my iltmus test for quality
<heat>
__sync_synchronize is way overkill
<heat>
mjg, what do my locking primitives say
<mjg>
what does it expand to? a full barrier?
<heat>
yes, mfence
<mjg>
heat: i already told you why they suck
<mjg>
wow, that's stupid
<mjg>
xchg already provides a full fence
<heat>
mjg, yes, just wondering what they say about my OS
<heat>
"passable but mildly incompetent" is probably a solid conclusion
<mjg>
that it ouperforms openbsd at best
<heat>
and net
<mjg>
this is the real crime: asm volatile("movl $0, %0" : "+m" (lk->locked) : );
<mjg>
with the comment above it
<heat>
why?
<mjg>
stackoverflow-level disinformation
<heat>
wait, why does it do a memory fence before the store
<heat>
in fact, you don't need a memory fence here at all
<mjg>
you don't
<mjg>
on that cpu
<mjg>
it is common to not know that bit though
<mjg>
you *do* need it on everything else
<heat>
do you need a full one tho?
<heat>
or just a store memory barrier?
<mjg>
"release" it is called
<heat>
yes so I think it's just a store memory barrier
<heat>
"Creates an inter-thread happens-before constraint to acquire (or stronger) semantic loads that read from this release store. Can prevent sinking of code to after the operation."
<mjg>
it guaranatees or *loads* and *stores* earlier in program order are finished
<mjg>
at the same time does not prevent ops *past* the fence from leaking up
<heat>
fun fact: which in x86 linux store fences literally just expands to something like (addl $0, -8(%esp))
<mjg>
s/or//
<mjg>
i think you mean a full fence
<mjg>
i don't remember if the 32 sucker has the same trick
<heat>
although TIL there's an alternative there for the fence instructions
<mjg>
that's not smp_*
<mjg>
time to head off, happy to flame tomorrow
<heat>
it seems that smp_rmb and smp_wmb are just compiler barriers
<heat>
huh
<heat>
hmm do you really need that lock there?
<mjg>
they are for amd64
<mjg>
i don't know about i386
<mjg>
for a smp_mb you do
<heat>
why don't you for a mov $0, (lock) then?
<mjg>
see the previous remark about ops leaking up
<heat>
for RELEASE semantics at least
<mjg>
there is no lock mov
<mjg>
or at least i have not seen one :]
<mjg>
burp really need to go dawg
<heat>
ah so mov works differently here, got it
<mjg>
mov is literally just store this shit over there
<mjg>
so does not matter what other cpus are doing in the area
<mjg>
in contrast smoething like 'addl' could lose existing state
<mjg>
which is why for proper smp use it gets the lock prefix
<mjg>
and lock addl 0 is a no-op in terms of content of the target area
<mjg>
while still resulting in all synchro normally associated with a locked op
* mjg
off
* heat
nods
Vercas has quit [Remote host closed the connection]
Vercas has joined #osdev
<chibill>
<heat> "except in x86 because screw..." <- lol XV6 was targeting x86, they stopped that and rewrote it to target risc-v now.
<heat>
yes, yes they were
<heat>
my plans were more to write a simple early unix clone
<heat>
kinda started mixing up unix and xv6 when writing those sentences
catern has quit [Read error: Connection reset by peer]
<kof123>
" cvs where there's six self checkout stations and one guy to tend to them / greet guests" <squints eyes, not familiar enough with cvs to tell if double-joke about cvs checkouts>
<epony>
sixty-six
<epony>
mirrors
<epony>
of 1 main CVS server
<epony>
and 1 backup CVS server
Vercas8 has joined #osdev
<epony>
figure it out, beats github 100% of the time since 1991
gabi-250_ has quit [Ping timeout: 255 seconds]
Vercas has quit [Ping timeout: 255 seconds]
gildasio has quit [Ping timeout: 255 seconds]
Vercas8 is now known as Vercas
<geist>
i was gonna say but but cvs has some nice properties... but then i can't think of them
gabi-250_ has joined #osdev
gildasio has joined #osdev
<chibill>
CVS = Customer Values Separated
<epony>
text format of the metadata / repo files
<epony>
and known "operation" / generic support everywhere
<epony>
simple enough to be implemented as a yak-shaving task
<epony>
and suitable for coherent development teams