<mrvn>
clever: setting A/D bit should tell the cpu you don't care. they can't get any more set.
<clever>
mrvn: that works, but how is the cpu to notice when D has been cleared?
<mrvn>
clever: some form of INVLPG
<clever>
but then again, i might clear D when i put it into swap-cache
<clever>
at which point, i would RO the page, and yeah INVLPG
nyah has quit [Quit: leaving]
<mrvn>
Is anyone using hardware dirty anyway? Don't you do it in software so you can maintain your LRU list? Or do you periodically scann the page table and update the LRU in intervals?
<clever>
ive not done anything more then enable d-cache with the mmu
<mrvn>
I guess you have to go through the page tables and clear the A bit periodically or the LRU list for accessed pages won't work. At that point you could check bot A and D bits.
[itchyjunk] has quit [Ping timeout: 252 seconds]
<mrvn>
Not having swap I never looked into how to handle the A/D bits for that.
ephemer0l is now known as GeneralDiscourse
<mrvn>
clever: do you set pages RO when you schedule them to be written to disk so they don't get modified on the fly?
<clever>
yeah, i think you would need to RO them first
<mrvn>
Note: linux does not which leads to raid1 getting pages out of sync on the disks.
<mrvn>
but only blocks no longer in use by the FS or so the devs claim.
[itchyjunk] has joined #osdev
<mrvn>
example: page is dirty, write to disk 1 starts, page is modified marking it dirty again, page is written to disk 2, file is deleted, page is never scheduled for write back again.
<clever>
oops!
<mrvn>
then the monthly raid check gives errors because the disks disagree.
<clever>
and without checksums, you have no way to know which is right
<mrvn>
and without goign through all the used blocks of the FS you have no idea if that error is relevant or not
<mrvn>
and yes, it's not repairable. So the same error shows up every month unless the block gets reused by the FS.
<mrvn>
This is also a flaw in the separation of FS and raid I believe. Say it is a used block. On read you get the block from a random disk. The FS sees an checksum error. There is no way for the FS to ask the raid for the other copy of the data to see if maybe that is better.
<mrvn>
or to restore the block from parity in case of raid5/6.
<clever>
yeah
<clever>
zfs solves all of those issues, checksums added, mirroring, parity, and fs
<mrvn>
clever: yeah. It still has the pools and the fs separate but it has enough knowledge so the FS can interact directly with the pool for this
small has joined #osdev
rei-ayanami has joined #osdev
<rei-ayanami>
Using tcc to compile linux kernel? Is it possible or does it not support enough of the gnu stuff, working on minimal linux from scratch with emacs as the user shell. would prefer self compiler
<mrvn>
Iirc the device-mapper raid1 can be combined with the checksum target.
<clever>
mrvn: yeah, basically, a pool contains many datasets, a dataset is just an id# -> object mapping, and an object is just an array of fixed-length records
<clever>
and then all metadata is just more records
<mrvn>
rei-ayanami: try it. I highly doubt it.
<mrvn>
rei-ayanami: 2 problems to master: 1) compiling at all, 2) does it work?
<rei-ayanami>
mrvn ill give it a try and see what happens
<rei-ayanami>
at the worst a fun learning exercise
<zid>
llvm can barely manage it :P
<zid>
it needs a *lot* of gnuisms, especially the asm() constraints
dutch has quit [Quit: WeeChat 3.8]
fedorafan has quit [Ping timeout: 248 seconds]
fedorafan has joined #osdev
<rei-ayanami>
I guess I'm confused as the osdev wiki says that tcc has inline assembly in gnu style, but I've heard it can't compile linux kernel
<rei-ayanami>
``TCC supports inline GAS syntax assembly like GCC: ''
<rei-ayanami>
What are my options if linux is too bloated as a kernel for me? Only thing I need to do is run emacs, don't need X
<kof123>
if this is the tcc i am thinking of, then maybe 2.x-ish it could compile the linux kernel (or with patches)
<zid>
what does X have to do with the kernel
<zid>
don't build the drm layer if you don't want the 10kB of wasted memory it causes
spikeheron has joined #osdev
<rei-ayanami>
zid idk if it does honestly, assuming that not requiring X or wayland might reduce some kernel requirements
<zid>
so you're just.. guessing? O_o
<rei-ayanami>
Yes.
<zid>
also if you build it with tcc it will be noticetably worse
<zid>
so you're ass-backwards
<rei-ayanami>
Weird and disrespectful reply, you're /ignore-d
<zid>
"I am mad because someone noticed I was wrong"
<rei-ayanami>
Thanks, but I'll be sticking with tcc.
<geist>
rei-ayanami: yeah sorry, he does that. i think he means well
<zid>
I didn't do shit gist
<zid>
stop sticking up for arseholes at my expense thanks
<geist>
well i agree with them in general
<geist>
ie, it's fun to rib on each other because we know each other but when someone new pops in it's generally nicer to be not as... ribby
<zid>
I didn't rib anybody over anything
<rei-ayanami>
It's like that on every irc channel, dw
<zid>
I asked if they were just guessing or if they knew something I did not, then pointed out using tcc and their stated goal are severely misaligned.
<geist>
uh huh
<zid>
If you have something to say say it
<geist>
the tone matters zid
<zid>
what tone?
<zid>
casual?
<geist>
the tone that you're apparently completely unaware of
<zid>
okay then so, what tone?
<geist>
again. i'm not going to argue with you about it
<geist>
because it's pointless, you'll just argue ad nauseum until i disengage because i aint got time for that
<zid>
so you're just doing it to tease me k
<geist>
nope
<zid>
well if you're not going to tell me, then yes you are
* geist
shrugs
<zid>
and we're back to: please stop doing things at my expense
<heat>
yay #osdev drama
<heat>
rei-ayanami, linux requires a lot from its compilers
<heat>
probably not trivial to compile using tcc
small_ has joined #osdev
<samis>
consider that making *LLVM/Clang* compile linux was something that took a bit
<geist>
are you trying to port it to a new architecture? tcc is a usual first place to do that
<zid>
yea I did mention the llvm
<zid>
The output would also be utter dogshit, in exchange for all that effort
small__ has joined #osdev
<rei-ayanami>
heat is there an alternative kernel I can use that's less demanding for what it needs from the compiler
<mjg>
i note inline asm is not the only potential problem
<geist>
though it's possible the porting to llvm/clang actually make linux a bit less gnutastic. maybe
<heat>
rei-ayanami, probably netbsd?
<mjg>
there are numerous extensions which may or may not have fallbacks
<zid>
mjg: yea 'lots of gnusisms' is the main one, like I said
<heat>
i assume netbsd is quite a bit smaller
<heat>
or openbsd but NO
<geist>
xv6 maybe
small has quit [Ping timeout: 260 seconds]
<zid>
zfs people currently doing bizzare things with addresses of local labels which isn't even valid in any dialect and moaning to gcc about it :p
<rei-ayanami>
geist not a new architecture. I'm just doing a custom OS (was presuming to do gnu+linux but seems that may be more than I need) and want to have a suckless tiny compiler because gcc seems bloated
<heat>
yeah but for emacs?
<geist>
depends on what the purpose of the compile is for
<mjg>
geist: that i would that. llvm keeps adding numerous externsions to act as a drop in replacement for gcc
<clever>
zid: that sounds like goto*
<mjg>
i just rmembered that they made asm goto mandatory
<heat>
geist, don't think xv6 will run
<heat>
will run emacs*
<mjg>
or at least planned to
<mjg>
i would deeply surprised if tcc knew how to do it
<rei-ayanami>
Should I just cross-compile the linux kernel and use the binary if I don't want to have to deal with gcc on my system?
<zid>
clever: interprocedural computed goto
<heat>
rei-ayanami, sure
<rei-ayanami>
I'll still try with tcc just for fun
<geist>
yah if nothing else it might be educational
<samis>
the guix people do an extensive bootstrap but i don't think they build linux with the bootstrap compilers
small_ has quit [Read error: Connection reset by peer]
<rei-ayanami>
I think it's cool guix uses a guile process manager
<rei-ayanami>
I wonder if I might want to use that...
<geist>
though i guess you could run a series of scripts that sets thigns up and drops permissions and then execs emac
<rei-ayanami>
Meh maybe not the largest but I still don't want some randos unaudited tarball from elpa running with that level of privileges
<mjg>
too bad they did not compare perf of compiled code
<mjg>
or quality of warnings generated
<rei-ayanami>
geist yea I think as OS learning proj will try to implement more syscall safety for emacs as pid 1
<mjg>
i have hard time imagining tcc does more than a small fraction of analysis
<geist>
yah i get the impression that the average compiler spends a nonlinear amount of time as the optimizations get more complicate
<geist>
even from running say a gcc 2.x or 1.x series and comparing it with modern gcc on the same code
<geist>
it's orders of magnitude different run time wise
foudfou has quit [Quit: Bye]
<heat>
mjg, that is not the point
<mjg>
where
foudfou has joined #osdev
<heat>
tcc is not competing with any other toolchain
<heat>
it's just a minimal compiler, that's its thing
<mjg>
so what is the significance of bench numbers posted
<heat>
"fast" but shitty
<geist>
i was just talking about the bencharmks that are on the tcc page
<heat>
cuz it is fast
<heat>
just not any good
<geist>
but yeah it's not a super serious rigorous thing. probably more to demonstrate that it's simpler if nothing else
<mjg>
if tcc is just a troy project, good for the author, but maybe they should note this is not apples to apples
<geist>
someone mentioned tcc 2 though, is that a later derivative of this?
<zid>
I mean, it's sort of in the name mjg
<zid>
It's a bit like making the world's largest pizza, nobody's doing it for the taste :P
<rei-ayanami>
kof123 I think
<geist>
kinda hard to tell what the future is, since the i guess official tcc page is this thing where it right off the bat says 'i'm not working on this anymore'
<geist>
like, if there is some sort of current development, should at least point the url at it
<rei-ayanami>
says to check the mailing list lol
<rei-ayanami>
annoying imo
<heat>
tcc does not support asm goto, so that's a no-go out of the bat
<geist>
yah, oh well, probably winding down. if i could find the git repo could at least see where the dev is (if it has one)
<heat>
if for nothing else
<zid>
I think git probably postdates tcc development significantly
<geist>
also i guess pretty soon you might need some rustc action to build linux
<heat>
cry.jpeg
<zid>
I hope not
<zid>
I might even plan hardware purchases around not having to need rustc
<geist>
i did grep around in the linux tree the other day to see what sort of .rs stuff existed. so far at the time i only saw some shim .rs library stuff
<geist>
things like kprint!() style stuff
<heat>
yeah there isn't much AIUI
<zid>
The near future plan as far as I know is just to allow some x86 drivers to be written in it
<geist>
so i'm guessing somewhere there's some branch where a more complete layer is being constructed
<zid>
because rust isn't portable outside of x86
<geist>
since presumably they'd need at least wrappers around some sort of common set of apis
<heat>
i think most of the work is being done outside the main tree
<heat>
there has been an nvme driver for quite some time as a PoC
<geist>
yah i guess some module could provide its own layer basically
<zid>
drivers however are like.. the least useful place to use rust
<heat>
you only know what layers you need by writing some PoC stuff (or actual needed code)
<zid>
unsafe { entire driver } :P
<heat>
geist, rando question but are you volatile-ing or WRITE_ONCE'ing your mmu code?
<geist>
former but should be doing the latter
<geist>
at least in some places
<geist>
actually fun thing we discovered today: the gic driver was having a problem in zircon because of the aforementioned stnadard volatile register access
<geist>
turns out on ARM64 on KVM it's quite strict about eactly what instruction you can use
<clever>
whats WRITE_ONCE do?
<heat>
what's the fuchsia WRITE_ONCE? __atomic_store with relaxed memory mode?
<geist>
(which makes sense in restrospect)
<heat>
clever, makes sure the compiler doesn't fragment your store or something
<geist>
specifically it emitted something like `std xzr, [x0], #4`
<clever>
ah
<heat>
or reorder it
<geist>
KVM just kills qemu on that
<Mutabah>
zid: "because rust isn't portable outside of x86" - ... and riscv and arm
<geist>
one of these 'how did we get along with this thus far' because specifically KVM only seems to want to handle ARM instructions that trap into EL2 with assist
<Mutabah>
(When you're doing no_std, any platform supported by LLVM will work)
<heat>
IIRC/AIUI it does something like #define WRITE_ONCE(var, val) *((volatile Type*) &var) = val;
<geist>
which explicitly calls out that load/stores *without* writeback
<geist>
so actually in the case of fuchsia theres a lib deep in it called mmio-ptr (really just a header file with some inline asm) that explicitly wraps a bunch of accesses with fixed instructions
<clever>
i forget, how does cache maintaince work with arm hypervisor?
<clever>
can the guest flush caches?
<geist>
so it's not only write-once, its exactly a known instruction
<zid>
Mutabah: riscv isn't a platform, and arm is just x86 for hipsters (this is a joke)
<clever>
does the cache control flags in the guest page tables still work, for mmio windows to not cache?
<geist>
and it's different per arch, obviously
<zid>
llvm also supports what, playstation 1?
<kof123>
no i meant linux 2.x :D
<geist>
clever: there's some control bits up in EL2 that specify what happens to lower level cache flushes
<heat>
zid, the only actual relevant archs are x86, arm, arm64, riscv
<heat>
and /maybe/ PPC
<geist>
generally the idea is that you squash invalidates into something like clean+invalidate
<clever>
ah
<zid>
two arms but only one x86? bias.
<geist>
since that might 'uncover' data that the guest shouldn't see
<geist>
otherwise as long as the guest can't see through to stale data it can flush/etc
<zid>
linux is actually one of those rare software projects where it will actually get ran on more than x86 and arm
<heat>
zid, yeah because arm32 is still heavily used but i686 is as dead as margaret thatcher
<bslsk05>
seyko2/tccboot - Using TinyCC to compile Linux kernels (6 forks/55 stargazers)
<clever>
and invalidate could discard unwritten data from another context
<geist>
exactly
<clever>
what about ASID'd and the TLB across guests?
<zid>
heat: The final boss when we invade hell? I don't think i686 is that bad.
<geist>
there's a higher level VMID that the guest is running at (at stage 2 translation)
<geist>
so effectively each TLB entry is tagged with a VMID:ASID pair
<clever>
ah, so it acts like a second tier
<geist>
and thus doesn't leak
<clever>
yep, that solves it nicely
<heat>
oh and IA-64 + SuperH because no one wants to piss off the computer necrophiliacs cc mjg
<geist>
yah same on x86
<geist>
the vmid/asid stuff even looks the same at two levels: when loading a S1 page table context you load the 16 bit ASID, but the hypervisor when it sets the S2 page tables it instead lodas a 16 bit VMID in the same slot
<geist>
so it's pretty consistent. RV's stacked paging for hypervisor does basically the same thing
<zid>
heat: alpha, sparc, mips
<zid>
blackfin
<zid>
blah blah
<heat>
and yes SuperH by hitachi, the vibrator massage wand guys
<heat>
geist, anyway why are volatile PTs bad anyway?
<zid>
Their PC department used to be a lot bigger than it is now, a lot of the hdds I owned in the past were hitachi
<heat>
sounds like most of the accesses would be READ_ONCE/WRITE_ONCE anyway
<zid>
everybody sold their hdd dartpments oer the years though
<zid>
wd is the only one left
<Jari--_>
morning
<geist>
well not bad until they are. in general really good mmu implementations end up switching to atomic accesses for all the page tables anyway
dude12312414 has joined #osdev
<geist>
but it's generally not required until you have A/D bits fully wired up
<clever>
geist: i assume youve been following the usagi electric pdp-11 stuff?
<geist>
clever: yep!
<geist>
am a patron too
<heat>
geist, ugh atomic everything?
<clever>
nice
<heat>
i know linux just has spinlocks. i also have em
<clever>
i only discovered him when he bought a wang, lol
<geist>
heat: well, no its more subtle than that
<geist>
but for example if you are zeroing out a page table, you can and probably should do an atomic swap of the entry with 0
<geist>
then you can harvest any A/D bits that may have been written back simultaneously
<geist>
if it's the initial write of the entry you probably dont need to do that
<geist>
provided you know you write the entry in exactly one instruction, etc
<geist>
which using a volatile should do as long as it's just a simple word aligned
<heat>
does make me wonder if you can race with the TLB writeback?
<clever>
yeah, atomic update the entry, clear tlb, then check the old table, and see what happened before the tlb flush
<geist>
you hella can. that's why arm pretty much tells you to do this. also why break-before-make is required, to kill any races with other cores
<geist>
x86 being strongly ordered it's much simpler
<clever>
break-before-make is what exactly?
<clever>
if your changing a mapping, make it unmapped temporarily?
<geist>
kill entry; flush tlb globally; then put new thing in place
<geist>
right
<geist>
it's to avoid other cores modifying A/D bits in the *new* entry because they're weakly ordered
<geist>
(an ARM thing)
<clever>
ah
<Mutabah>
Sounds like an electronics term (make sure that conflicting switches are off before transitioning - otherwise you might have a transient short circuit)
<clever>
reading from the old page, but writing to the A/D bits of the new entry
<geist>
and when breaking up a large page with smaller ones. you have to flush out all the old large pages from all the other cores before you start filling in newer smaller ones
<geist>
so you dont end up with a conflicting TLB abort, which is Bad
<geist>
Mutabah: yah also vague memories of karnaugh maps and avoiding glitches
<geist>
iirc you added extra links to avoid islands in the map so that no transition goes through a non state
<clever>
but i would also say you have bigger bugs, if another core is trying to read a page as the page is getting remapped by choice (mummap+mmap)
<zid>
Mutabah: That's why you get nice chips which have their input and output switching speeds be different
<zid>
they disable fast and enable slow
<clever>
but, i could see it happening not by choice, by the kernel shuffling physical pages around without telling userland
<geist>
yeah exactly
<clever>
thats another case where i would definitely want to break first, so the page doesnt get modified mid-move
<geist>
for user space in general the worst case there is another core page faults on something being shuffled around, enters the kernel, then hits whatever spinlock/mutex/etc is protecting the page table, and waits for it to finish
<heat>
geist, have you tried building generic page table code for every radix arch?
<geist>
then when it finally looks it decideds there's nothing to do and retries the page fault
<clever>
yep
<heat>
at least the basics
<geist>
heat: no but i did try a new scheme for the riscv LK code where it uses a generic page table walker routine that takes a 'what do i do here' lambda that various routines use to implement their logic
<geist>
it came out surprisingly well, codegen wise, though i dont know if it's flexible enough to continue with
<heat>
yeah i know
<clever>
i might adjust that slightly, not hold the mutex while the page is being moved
<clever>
but have a state flag on that page, that says "go join this waitlist"
<geist>
you could i guess build some sort of even more generic 'how do i get to the next level' walker thing
<heat>
I was thinking of a linux-like thing
<geist>
but frankly that's not my kinda code
<clever>
so page faults unrelated to the migrating page, can still fault right away
<heat>
maybe still behind a pmap-ish interface
<geist>
ie, heavily using C++ style algorithmic stuff
<clever>
how often does a kernel need to walk the page tables?
<geist>
when being modified basically
<clever>
couldnt you basically just have a map, and unmap call, and give the mmu code an allocator
<geist>
and then it's up to 'when does the kernel modify page tables'
<geist>
that's basically what pmap style kernel mmu logic does
<clever>
ah
<geist>
it's an opaque interface for 'do whatever the arch needs to get it done'
<geist>
linux is the weird outlyer here where that's kinda inverted and core to the kernel VM design
<clever>
storing state in the page table and such?
<heat>
yes
<geist>
yah and in general assuming there is a page table and using it as a high level data structrue
<geist>
and for arches that dont have a radix style page table, they have to emulate it at some mid level
<geist>
ie, power
<heat>
and having a nice generic thing where you can do for_every_pml4(){for_every_pd(){for_every_pt(){...}}}
<geist>
or our old friend SuperH
<heat>
brrrrrrrrrrr
<geist>
(superh is a TLB miss style arch)
<heat>
or IA-64!
<heat>
HOW COULD YOU FORGET THE IA-64!
<geist>
or ultrasparc, or microblaze, or whatnot
<clever>
and there is also /proc/PID/pagemap, which basically exposes the lowest level of the paging tables as one huge file
<clever>
but it has a somewhat uniform cross-arch format i believe
<geist>
it's annoyingly super effective. it's a case where linux does a Bad Design that becomes the defacto solution
<heat>
what is?
<geist>
so defacto that basically x86 has turned into the template for all future architectures (like riscv) because it's too expensive to deviate from that design
<heat>
ah
<heat>
clever, pagemap isn't a straight page table dump
<geist>
OTOH i think history has proven that radix style page tables are pretty good
<clever>
heat: yeah, its been normalized, so it works the same way on every arch
<clever>
and anoyingly, i dont think pagemap includes mmio regions
<clever>
so if you mmap mmio via /dev/mem, it just shows up as 0's
<heat>
pagemap also has struct page info
<geist>
i thin there was a period there in the 90s where TLB miss style arches were considered quite powerful since you could be highly flexible in how you construct your TLB in software, could use lots of page sizes, etc. ie ia64 stuff
<clever>
i spent a month trying to fix an mmio bug
<clever>
and geist wound up solving that bug by accident, heh
<geist>
but since the world says a smallish amount of page sizes and radix tables are just fine. no real OS has a super good solution for anything else
<geist>
so the tail is wagging the dog somewhat
<heat>
oh yeah btw they're going to kill struct page
<geist>
but really i think the problem with TLB miss arches is they're hell on the pipeline. they were one thing when you had cpus that were running one instruction at a time, etc
<geist>
but then now having to take all these tlb miss exceptions all the time on modern cores is hard to predict through, etc etc
<geist>
oh yeah? re page?
<geist>
re flat physical page tables (ie, ia64 small table i think and POWER) supposedly have some NUMA issues
<heat>
yeah, aiui they're making most of the state a thing when you allocate instead
<geist>
because you have to blat it in physcal ram somewhere, so by definition it's closer to some cores than others on a NUMA system
<clever>
geist: one weird thing i saw with the centurion, is that the cpu basically has 16 modes, and each mode has its own bank of registers (like FIQ), could that allow servicing a tlb miss handler, in a kind of hyperthreaded manner?
<clever>
where you have a second bank of registers, and a pseudo core, that only exists to run tlb-miss handlers
<heat>
and struct page will just be essentially a struct page {struct folio *allocated_state; // folio represents more than a page, and holds the state};
<clever>
and the pipeline can interleave normal and handler opcodes at the sign of a future fault
<geist>
clever: yeah iv'e seen various arches like that, just nothing survived to the modern day
<clever>
rather then halt the world, and change modes
<geist>
was fairly common to bank all the regs for differnet modes back when yuo had just a handful of regs
<clever>
yeah, its only got something like 5 registers
<geist>
even something like z80 has that. two register sets, iirc
<geist>
but then there's only a few registers
<clever>
and i think 256 bytes of registers total, for every mode, they are even memory mapped
<geist>
nother one i remember was a whacky little embedded network processor in the 2000s. it had iirc like 8 or 16 register sets and you could configure if it round robined between them (ie, a barrel SMT thing) and/or you could pin some of them to be tied to a particular irq level
<geist>
such that when it entered IRQ X it'd effetively just start running a thread that was parked and ready to go
<geist>
hmm, what was that called...
<clever>
ah yeah, centurion irq's are also similarly weird
<clever>
each irq forces the cpu to switch to a certain banked set or regs's, and just resume at whatever the saved PC was for that bank
<clever>
so its more like each irq handler, is a while (true) { ...; eret(); } loop
<geist>
yah. not a bad idea, but probably not really needed for something like that, but also it wasn't really a success
<geist>
so it's not like it was necessarily a *good* design
<clever>
and the timer irq is special, in that the timeout is in a register banked to the timer irq mode
<clever>
and it just jumps into that mode upon hitting 0
<clever>
so that register is just counting down on its own, without even being in the active mode
<geist>
Slingbox. that was the product i was thinking about that used one of these little embedded things
<geist>
was an ahead-of-its-time network streaming box
<heat>
guys guys guys, what if we had a banked register set and used to to SWITCH TASKS?!?!??!
<heat>
craaaaaaaaaayzeeeeeeeeeee idea right?
zid has quit [Ping timeout: 264 seconds]
<\Test_User>
with registers being as slow as ram or with a severe limit to number of tasks
<\Test_User>
(as slow as bc they'd be in ram)
<geist>
yah the old TMS9900 solution
<clever>
\Test_User: one common theme i keep running into with centurion and pdp-11, is that the ram is actually faster then the cpu
<geist>
point a register at memory and whatever was there became the register set
<\Test_User>
clever: true, for slower cpus it'd make sense
<clever>
and moving data from cpu<->ram is the slow part
<geist>
yah there was a period there before cpus outran memory and kept going where you could use ram that way
<clever>
in the pdp-11 with the CD bus, the ram controller is actually dual-ported
<clever>
so both the cpu and dma can be hitting ram at the same time, and the ram is fast enough to serve both
zid has joined #osdev
<geist>
yah if you want to read about various DEC busses there's a pretty good wiki that talks about a lot of them
<clever>
ive been focused on the centurion and hawk lately, trying to make an emulator
<bslsk05>
gunkies.org: UNIBUS - Computer History Wiki
<geist>
i forget which bus thei particular PDP/11 he has uses
zid has quit [Read error: Connection reset by peer]
<geist>
i think it was a later cost-reduced version where the cpu was basically an ASIC (LS11 I think)
<clever>
usagi's pdp-11 backplanes have SBUS and CD
<clever>
CD is a dedicated interface between the cpu and ram, and nothing else
zid has joined #osdev
<zid>
I like the way that if you reboot my modem, it connects, then disconnects, then connects again, and that I never remember.
<clever>
and a CD capable ram, is on both the SBUS and CD bus (as is the cpu)
<clever>
so then, mmio and dma happens over SBUS, but cpu<->ram happens over CD, and they can run in parallel
<zid>
My new way to handle IRQs is that every irq handler pointers to the same 'ret' instruction, and rsp0 is loaded with a rop-chain
<clever>
so the memory card is effectively functioning as dual-port ram
dude12312414 has quit [Quit: THE RAM IS TOO DAMN HIGH]
small has joined #osdev
small__ has quit [Ping timeout: 252 seconds]
<mrvn>
zid: my IRQ handler just retruns the "wait-for-irq" message to sender.
<mrvn>
it's a bit backward but drivers basically poll for IRQs. Sending the "wait-for-irq" message enables the IRQ (+EOI on x86) and when it happens it gets masked and the message returns.
<mrvn>
How do you create the rop-chain?
gorgonical has quit [Ping timeout: 248 seconds]
elastic_dog has quit [Remote host closed the connection]
elastic_dog has joined #osdev
heat has quit [Ping timeout: 252 seconds]
[itchyjunk] has quit [Read error: Connection reset by peer]
<ghostbuster>
rop chains have uses other than exploits?
<moon-child>
zid: you jest, but there is a genuinely useful version of that which mjg and I came up with independently: you want the isr to malloc, but it can't do that if somebody is currently mallocing; so you check if somebody is mallocing, and if so overwrite the return address to run the isr as soon as malloc is done
<moon-child>
then you don't have to sti in malloc
<moon-child>
ghostbuster: ^
<zid>
I never jest.
<ghostbuster>
neat
elastic_dog has quit [Ping timeout: 260 seconds]
<moon-child>
(obviously this generalises to arbitrary values of malloc. And in the limit, it's a scheduler unto itself; but before the limit, it's a neat, simpler mechanism)
<zid>
People suggested I was jesting when I said I was going to store all of my data by sending it inside pings to random machines
<rei-ayanami>
`` TCCBOOT is a boot loader able to compile and boot a Linux kernel directly from its source code. TCCBOOT is only 138 KB big (uncompressed code) and it can compile and run a typical Linux kernel in less than 15 seconds on a 2.4 GHz Pentium 4.''
<bslsk05>
seyko2/tccboot - Using TinyCC to compile Linux kernels (6 forks/55 stargazers)
<rei-ayanami>
``Project considered as finished. tcc is quite good to compile linux-2.4.37.11 kernel with PCnet-PCI II ethernet driver. A telnet from tccboot to other host is possible.''
<rei-ayanami>
this seems comfy
<rei-ayanami>
ok thats enough computer for one day
<kof123>
i linked to that :D
<kof123>
got buried in the other chat :/
<rei-ayanami>
kof123 i think must have missed it sorry friend
<kof123>
np, it was on slashdork IIRC
<rei-ayanami>
have you used it?
<kof123>
no, just remember it being "announced"
<kof123>
im not that old, but ...
<kof123>
there was a brief period bsds wanted to revive pcc
<rei-ayanami>
seems very useful but i dont know why i'm seeing virtualbox stuff in the git commit messages
<kof123>
probably around the same time
<kof123>
well, it is a horrible idea, but i am interested in bundling a compiler (or similar) for something jit-like perhaps
<kof123>
"reflective c" as it were
<kof123>
so any lightweight compiler interests me for that reason
<rei-ayanami>
my main critique is i don't want to use this random person's 2.5 kernel config from 6 years ago, i'm just gonna start try compiling modern minimal kernal with tcc and if it breaks check out their repo
<rei-ayanami>
yeah that seems pretty cool
<rei-ayanami>
tcc can be used to run scripts as fast as executable apparently
<rei-ayanami>
i want to run emacs as pid 1 so my idea is probably worse
<rei-ayanami>
definitely cooler than gcc, hoping i can just skip gnu all together
<kof123>
well, something like this as well (which is also ancient): C-Mix: Making Easily Maintainable C-Programs run FAST The C-Mix Group, DIKU, University of Copenhagen "program specializer" you should be able to find a pdf 10.1.1.37.655.pdf maybe with some simple examples
<kof123>
well i think there was a systeme ...im unsure how serious it was ...
<kof123>
and a chatter with the nick emacsomancer "so what, are you god now?"
<kof123>
its not unheard of :D
<rei-ayanami>
interesting
<kof123>
* so, what, are you
<geist>
kof123: oh the NERV
gog has quit [Ping timeout: 260 seconds]
_whitelogger has joined #osdev
elastic_dog has quit [Ping timeout: 252 seconds]
elastic_dog has joined #osdev
rei-ayanami has quit [Ping timeout: 260 seconds]
Matt|home has quit [Ping timeout: 248 seconds]
small has joined #osdev
bgs has joined #osdev
hmmmm has quit [Ping timeout: 248 seconds]
hmmmm has joined #osdev
elastic_dog has quit [Remote host closed the connection]
elastic_dog has joined #osdev
bgs has quit [Remote host closed the connection]
danilogondolfo has joined #osdev
GeDaMo has joined #osdev
slidercrank has joined #osdev
gildasio has quit [Ping timeout: 255 seconds]
gildasio has joined #osdev
jjuran has quit [Quit: Killing Colloquy first, before it kills me…]
jjuran has joined #osdev
gog has joined #osdev
nyah has joined #osdev
onering has joined #osdev
Beato has quit [Ping timeout: 256 seconds]
fedorafan has joined #osdev
<netbsduser`>
just had a look at qemu's m68k "virt" machine, looks like it might be a nice target
<dinkelhacker>
Hey guys, would you say it is a good idea to have seperate physical memory regions where only kernel pages or only user pages go? Or would it be better to just say: That is all the memory I have, and I'll just allocate pages no matter where the request is comming from?
<zid>
doesn't matter
<zid>
sounds like a royal pain to do it that way for no benefit
<zid>
what do you hope to gain by segregating physical memory? That's already the job of the mmu/memory protection
<clever>
rowhammer maybe?
<dinkelhacker>
No security concerns regarding my toy os :D
<dinkelhacker>
No I was just thinking if there might be a benefit of having guaranteed memory for kernel space. You know, like taking one gig out of all the memory and making that exclusive to the kernel. On the other hand that could still be done on the cirtual memory level without seperating the physical memory
<dinkelhacker>
*virtual
<zid>
still means tagging all your memory allocations as 'for' userspace or 'for' the kernel, so still sort of a pain
<zid>
but at least semi-useful
<mrvn>
dinkelhacker: I recommend just handling phyiscal memory as a stack of 4k objects and don't even consider anything bigger or physical continuity unless you have hardware that absolutely needs it (in which case you reserve memory for that at boot).
<mrvn>
zid: For RPi you have to do it because the VC can only access the first 1GB. On x86 you might have 32bit PCI. It's not a user/kernel split but split by intended use.
<mrvn>
s/you have to do it/you might have to do it/
<dinkelhacker>
zid: I mean you have to treat the allocations different anyway, right? Regarding access flags and the likes.
<mrvn>
dinkelhacker: it's not uncommon to reserve xMB memory and user space allocations fail when free < x already.
<zid>
yea true
<dinkelhacker>
mrvn: I think I'll just start with what you said and keep it simple for now.
mavhq has quit [Ping timeout: 268 seconds]
<mrvn>
dinkelhacker: don't do anything fancy for physical memory. Just make a linked list out of all the free pages and keep a count how many there are. Hint: you store the next pointer in the page itself.
<mrvn>
Orchids are interesting that way, they frequently use humans for pollination. :)
<zid>
vegetables are evil
<mrvn>
zid: no bugs, no veggies. What is left to eat?
* mrvn
feeds zid artificial sweetener and food coloring for a week
mavhq has joined #osdev
foudfou has quit [Ping timeout: 255 seconds]
foudfou has joined #osdev
gabi-250 has joined #osdev
pg12 has quit [Ping timeout: 265 seconds]
wand has joined #osdev
pg12 has joined #osdev
gabi-250 has quit [Remote host closed the connection]
gabi-250 has joined #osdev
bgs has joined #osdev
gabi-250 has quit [Quit: WeeChat 3.8]
<sakasama>
No bugs? Computers are now delicious.
<mrvn>
lets make an optical computer out of suggar crystals
<GeDaMo>
Is sugar a semiconductor?
<mrvn>
I said optical, not electrical
<GeDaMo>
Ah
* sakasama
would eat it either way.
<GeDaMo>
Me too :|
<FireFly>
that's great, finally a way to convince people to recycle their old electronics :p
<FireFly>
if you can just eat it as a snack
clever has quit [Quit: leaving]
<dinkelhacker>
I have another question regarding that whole early mmu topic. Currently I am compiling the kernel with virtual adresses (no pic).In the initial assembly code I'm setting up a 'early va mapping' where I identity map the phys addr space and map my kernel va space to where the kernel got loaded. That's pretty standard I guess. However, this is just possible because I know how much memory the target
<dinkelhacker>
has. If you would want to read that dynamically from the dtb, whould that mean that you have to parse the dtb from assembly?
<heat>
no
<heat>
the standard way to address this is to map only what you need
<heat>
imagine your kernel goes up to 8MiB max (in size). you map the first phys_base + 8MiB
<mrvn>
a) you only need to map the kernel + dtb, b) you can map memory that doesn't exist.
<heat>
if you then need to look at physical addresses without having virtual addressing really up (later on, already in C), you can add a large linear mapping of physical memory
<heat>
it's the standard-ish solution
<mrvn>
You can just map 1GB around the place the kernel is to 0x80000000
<mrvn>
or the equivalent for 64bit
<dinkelhacker>
mrvn: Didn't get that last two messages. Why whould i want to map 1GB to that specific 0x80000...? Regarding a) how do you know how large the dtb is?
<mrvn>
no reason it has to be 1GB. It's just a nice round number.
<mrvn>
Without parsing the dtb you don't know how big it is. But it won't be larger than 1GB.
<mrvn>
If the dtb is before the kernel then you have an upper limit for it.
<mrvn>
If it's after you can just map lots of memory or map each page as you parse the dtb.
<dinkelhacker>
Oh there is no standard way how it is loaded? Like always after the kernel?
<mrvn>
wouldn't count on it
<mrvn>
I would just map all of one L3 page table using the biggest pages you can.
<mrvn>
Which turns out to be 1GB for 64bit, 2GB for 32bit.
<dinkelhacker>
Okay... yeah so if it's after the kernel. You just map a large chunk and since your not really allocation anything untill you have parsed the dtb and know how much memory you really have nothing can go wrong?
<dinkelhacker>
Why an L3 table? I could just map one entry of an L1 (4K granule) which is also 1 GB?
<mrvn>
you can also identity map 1GB and only the kernel in higher half and parse the DTB from the identity mapping.
<dinkelhacker>
true
<dinkelhacker>
Thx mrvn, heat :)!
<mrvn>
Tip: Add 64k or so to the .bss section of the kernel to initialize your memory pool with some free pages. Then you can allocate stuff while parsing the DTB and then add all the unused memory to the pool.
<dinkelhacker>
speaking of it. What you said earlier about the linked list that contains all the physical pages. I mean if you know how much memory you have you can just put all the page structs in a big array at compile time and have it in the image. But if you do it dynamically wouldn't you need an 'early_kmalloc' that just gives you raw memory to put your list of pages in?
<dinkelhacker>
because otherwise it's a bit of a catch22?
<heat>
yes
<heat>
i have a whole bootmem allocator thing
<heat>
my page allocator and struct page allocation just grabs a big chunk of memory from the bootmem allocator
<bslsk05>
github.com: Onyx/pagealloc.cpp at master · heatd/Onyx · GitHub
<heat>
bootmem allocator is in mm/bootmem.cpp
<dinkelhacker>
Cool! I'll take a look at that.
Matt|home has joined #osdev
<mjg>
ohnoesyx? i would leech from sortix instead!
xenos1984 has quit [Ping timeout: 252 seconds]
xenos1984 has joined #osdev
<mrvn>
dinkelhacker: when you parse the DTB you put the pages for each chunk of memory into the free list and then allocate the page struct in virtual memory.
<heat>
mjg, shuddup stupid go back to BSD where S stands for stupid
fedorafan has quit [Ping timeout: 252 seconds]
<mjg>
i'm looking down at ohnoesyx valley from my bsd hill
<mjg>
you should call it WebDevOS, cause it looks like it was written by one
fedorafan has joined #osdev
gog has quit [Quit: Konversation terminated!]
<heat>
oh yeah
<heat>
you should call freebsd shit, because it looks like it was written by shit
terminalpusher has joined #osdev
<sham1>
Free as in free shit
<sham1>
Don't mind if I do
<sham1>
And of course it looks like a WebDevOS, it's C++ after all
<heat>
hey, only I can slag off C++
foudfou has quit [Ping timeout: 255 seconds]
wand has quit [Ping timeout: 255 seconds]
foudfou has joined #osdev
<sham1>
Hmm, I probably shouldn't be the one talking. I spend my days writing Java after all for work stuff
<heat>
EnterpriseAdapterVisitorOS
<mjg>
org.osdev.CustomOs.Factory
wand has joined #osdev
gildasio1 has quit [Ping timeout: 255 seconds]
gildasio1 has joined #osdev
<sham1>
Nah, the real fun begins when you start doing dependency injection to load drivers
gog has joined #osdev
<heat>
osdev in eclipse sounds sweeeeeeeeeet
<zid>
people seem to really love eclipse
<zid>
I've never even seen a picture of it
<zid>
never having owned a java
<gog>
i don't
<gog>
i hated it
<gog>
i hate visual studio too
<gog>
i hate everything
* gog
grumpy cat
<heat>
vscode nice
<heat>
people that say visual studio and vscode are the same thing are wrong
<FireFly>
eclipse was like, fine, when I used it 15 years ago
<FireFly>
god it's been a while
<heat>
eclipse sucks
<heat>
intellij idea is the best I think
xenos1984 has quit [Ping timeout: 248 seconds]
<FireFly>
might well be the case
<FireFly>
never really used any jetbrains tools, but I _have_ heard good things about them
<FireFly>
as far as IDE-shaped things go
<heat>
yeah
<heat>
i think clion is good too
<heat>
i should give it a try but im too deep in the vscode plugin rabbithole
<heat>
>Installed: 83
<zid>
I've never worked in anything where an IDE would be useful
<zid>
so vscode has passed me by so far too
<FireFly>
tbh life is pretty good now that we have LSP (which afaik is thanks to vscode, so at least that's nice)
<sakasama>
Each plugin I add to vim tempts me to delete them all.
<FireFly>
never had that many in the first place, I think vanilla vim with some config to fit preferences is enough for me
<heat>
i can't use vim
<heat>
it sucks
<heat>
i don't have enough proficiency with the thing to be bearably usable
<zid>
I can write text into it and save
<zid>
and that's 99% of what I wanna do
<sakasama>
I found vimtutor had the dual effect of basic proficiency with vim and irritation with everything else.
<sham1>
IntelliJ IDEA is indeed the gold-standard for Java IDEs
<sham1>
It's also incredibly cumbersome to use
<heat>
lol
<sham1>
It's like driving a tank to go grocery shopping
<heat>
well it's for True Enterprise Software
<sham1>
Mmm
<heat>
although I've never had issues with it in my hello world uni software
<heat>
i kind of like it
<sham1>
Like yeah, it works fine. And I'm not even talking about speed, but even stuff like moving around and interacting with the code feels annoying. Then again, I'm probably just too used to Emacs at this point
<heat>
praise be the unexec
xenos1984 has joined #osdev
dude12312414 has joined #osdev
dude12312414 has quit [Client Quit]
<sham1>
I actually had to use IDEA for a bit at work since the Java LSP server didn't work anymore for a while. I was so sluggish with it. Even tried the Vim bindings, and those also just felt off
<sham1>
But now I'm back to sane text editing
demindiro has joined #osdev
<demindiro>
Goddamnit, I thought I had a huge space leak but apparently stupid du is using 512 byte blocks instead of 4096
<demindiro>
Even though FUSE appears to be returning the correct block size, 4096, when I use stat on some random file
<demindiro>
du is ignoring the --block-size argument, wtf?
<demindiro>
--block-size does not do what I thought it did, sigh
<demindiro>
Doesn't appear to actually try to get the real block size anywhere
<heat>
demindiro, POSIX du blocks are 512 bytes long
<heat>
stat st_blocks are frequently 512 bytes long too
<heat>
(as in stat(2))
<demindiro>
So what's the st_bsize field used for then?
<demindiro>
Ah
<demindiro>
*optimal transfer size*
<demindiro>
bah
<heat>
yes
<heat>
st_blocks are in units of DEV_BSIZE
<heat>
dunno if this is a standard define
terminalpusher has quit [Remote host closed the connection]
terminalpusher has joined #osdev
clever has joined #osdev
<demindiro>
Okay so statfs does use st_bsize, but now that I check manpage for fstat it says "Number of 512B blocks allocated"
<heat>
demindiro, note that by default GNU du will show 1024 byte blocks
<heat>
you need POSIXLY_CORRECT=1 to get 512b blocks
<zid>
haha
<zid>
POSIXY_WOSIXY=1
<demindiro>
du is most certainly using 512 byte blocks for me
<demindiro>
no POSIXLY_CORRECT
demindiro has quit [Quit: Client closed]
<heat>
hmm, maybe it's just df
<heat>
du seems to report 512b here too
<zid>
people use du without h?
dh` has quit [Ping timeout: 268 seconds]
<hmmmm>
human readable is optional... ...because it was meant to be machine readable at first
<hmmmm>
perhaps
heat has quit [Read error: Connection reset by peer]
heat has joined #osdev
<kof123>
well, that makes sense. humans have powers of 2 digits, so it is perfectly orthogonal
<kof123>
holistic as it were
<heat>
du -smh is the only possible invocation of du ever
<heat>
the rest just fries the mobo
<clever>
i find `du -hc --max=1 | sort -h` to be very useful, sometimes add -x, depending on the use-case
bgs has quit [Remote host closed the connection]
divine has quit [Quit: Lost terminal]
divine has joined #osdev
GeDaMo has quit [Quit: That's it, you people have stood in my way long enough! I'm going to clown college!]
notdzwdz is now known as dzwdz
danilogondolfo has quit [Remote host closed the connection]
m5zs7k has quit [Remote host closed the connection]
gorgonical has joined #osdev
m5zs7k has joined #osdev
m5zs7k has quit [Ping timeout: 268 seconds]
m5zs7k has joined #osdev
<kof123>
cmix sort of works :) 721K cmix-2.0.12/bin/cmix 60K cmix-2.0.12/lib/libcmix.a. that's less < 1M, and then add ~12k of "shadow headers". had to change <iostream.h> to iostream and add some "using namespace std;" it does not like modern linux headers, but some of the examples work. it has its own internal "shadow headers" for libc (not sure if autogenerated, but looks like c89, + littered with its own pragmas). some of the sourc
<kof123>
s c++, so would have to have the ability to run that (this may be only the binary, the lib may be c). the binary generates a c program that uses the library, which, when compiled & executed (presumably passing the "known" value(s) as arguments) finally outputs a (c89?) "optimized" version of a function
<kof123>
*change iostream.h to iostream, and fstream, and ...
<kof123>
*modern linux libc headers
terminalpusher has quit [Remote host closed the connection]
gog has quit [Remote host closed the connection]
dh` has joined #osdev
<kof123>
a more practical usage than trying to jit with cmix might be, if you had C source to command-line utils laying around (and pre-configured etc. if necessary). then you could "optimize" a pipeline, write a script or shell that tries to optimize each program if possible
<kof123>
*user could specify which parts should be "optimized"