thinkpol has quit [Remote host closed the connection]
thinkpol has joined #osdev
dbana has joined #osdev
<geist>
Ah yeah that LK code is *old*
<geist>
Actually predates LK somewhat, i think i wrote it in mid 2000s sometime
<geist>
Really was only ever written against plain ext2. Needs some more features and cleanup.
<klange>
You probably don't need to do anything with some of those? I think some of the compat features are "hey this thing is here, but if you're read only you can ignore it"
<geist>
Iirc they have multiple flag fields
<geist>
One for ‘you need these features to read/write’ and another one for ‘you need this feature to just read’
<geist>
Like, yeah the different i node sizes, and whatnot is a read only mount breaker
<clever>
geist: i also notice, its not respecting the s_feature_incompat field
<clever>
those flags from the wiki i linked above, are all set
<geist>
In general there’s a question as to whether or not you want to be able to RO mount a journaled fs with an unplaced journal
<clever>
(on ext4)
<geist>
Un played
<klange>
RO_COMPAT_HUGE_FILE → Files (some?) have sizes in logical blocks rather than 512-byte sectors.
<clever>
> This filesystem has files whose space usage is stored in i_blocks in units of filesystem blocks, not 512-byte sectors. Inodes using this feature will be marked with EXT4_INODE_HUGE_FILE. (RO_COMPAT_HUGE_FILE)
<clever>
according to the wiki i'm reading
<klange>
RO_COMPAT_DIR_NLINK → The i_links_count in a directory can be reset to 1 if it has >=65k links.
<clever>
so its a per-inode thing
<geist>
Hmm, somewhere along the way they must have found space for the top 32bits of the file size too
<geist>
Since iirc early ext2 was 32bit only
<clever>
geist: thats probably INCOMPAT_64BIT
<geist>
At the time it was one of those things FreeBSD did better
<clever>
if you dont support INCOMPAT_64BIT, then you cant even do a ro mount, because youll mis-parse the huge files
<geist>
Yah
<geist>
Except the code currently ignores it, so it probably still mounts but will be off if its a large file
<clever>
my main goal is just to read boot files out of /boot
<clever>
without forcing the user to use ext2 for all of / (or a second /boot)
<geist>
Yah, traditionally for a long time lots of distros would format /boot as plain ext2 + SYNC mount options
<geist>
Presumably for boot loader compatibility
<clever>
ext2 doesnt get you much benefit over fat32
<clever>
one bad shutdown at the wrong time, and you dont boot anymore
<geist>
Well, that’s debatable, but yeah
<clever>
a journal should prevent that
<geist>
That’s why traditionally you mounted /boot with sync option
<clever>
if your bootloader can deal with the journal
<geist>
That’s the problem: read only mounting with a journal is problematic. What do you do with it?
<clever>
i dont know about the ext3/4 journal specifically
<graphitemaster>
Okay doing the whole mmap as a memcpy optimization actually is really fast after 128 KiB of data.
<clever>
but with sqlite, the journal has a backup copy of any blocks it was about to overwrite
<geist>
About the best thing you can do is parse it, build an in memory representation of blocks in flight, and then use a look aside list to consult journal blocks as you read things
<clever>
so if you re-route all reads, to the journal version, like an overlay
<geist>
Right
<clever>
then youll be viewing the rolled back version of the state, without actually applying it
<geist>
Generally it’s the rolled forward state in ext4
<clever>
ext4 writes the "new" block to the journal first?
<geist>
Essentially ‘imma bout to write these blocks’ (goes to write the blocks) (marks the transaction as complete)
<clever>
ahh
<geist>
Yah, it’s a forward journal
<geist>
That’s fairly standard and traditional. Not even that slow, since you can cache up a fairly large journal, and fairly lazily write it out and the real data
<clever>
sqlite has a complex dance, write the journal block without a header, sync, write the journal header, sync, overwrite the original blocks in the db, sync, destroy the journal header, sync, destroy the journal file entirely
<geist>
You just have these sync points where you have to ensure the journal is fully written before entering the next phase, etc
<clever>
sqlite also assumes that extending a file, creates garbage in the holes
<clever>
and other funky assumptions
<geist>
Yep. I left out the ‘update the journal superblock’ thing
<clever>
sqlite also combats the raw sectors, writing from either end
<geist>
I dunno exactly where ext4 puts that, but it’s probably just a single sector that says what the head of the journal is, since you can write that single sector atomically
<clever>
so if a partial write occurs, and the raw drive writes the sectors tail-first, sqlite handles it
<clever>
both the head and tail of a sector must agree, or the block is considered corrupted
<geist>
But anyway, the journal is a pain for embedded things. Even if you don’t play it back on mount you still have to track a fairly large amount of pending metadata
<geist>
Which may be difficult in a constrained memory situation
<clever>
the main case where i want ext4, is after i bring dram online
<clever>
so i'll have between 256mb and 1gig of ram
<clever>
klange: step 1, is to just falsely claim LK can read everything just fine, that is done
<clever>
step 2, realize LK lacks readdir() for ext2/3/4, so i have no way to test the next phase easily :P
<klange>
directories are for losers, Real Programmers know where the inodes for their files are
<klange>
in fact why do you even need a filesystem, REAL PROGRAMMERS know all the blocks constituting their files and just use them directly!
<clever>
klange: you dont use butterflies? :D
<geist>
Haha that was the LILO strategy
<graphitemaster>
inodes are for losers, Real Programmers know the physical location on the platter and can read it with a tiny magnet on the end of a needle without any drive controller.
<clever>
and also what a lot of ARM chips do for the bootup code
<clever>
rpi is a rare beast, where it handles MBR and fat32 in the bootrom, and loads from a file
<geist>
A fairly common strategy is to have a fixed run of sectors on NAND that you store the loader, etc. But since NAND can have bad sectors, it skips any bad ones and just reads the next
<geist>
But that’s easy to detect. As long as you overprovision a bit and probably store multiple copies of the binary, you’re cool
<clever>
geist: i think the bootcode.bin was just in the first 128kb of sectors on the raw nand, when booting from that
<clever>
ive not read that code closely, because i have no chips to wire up
<geist>
Yep. If it’s well designed it overprovisions, or has multiple copies
<geist>
Like 192KB to store 128KB, in case it has some bad nand sectors in the middle
<clever>
when booting from SPI, its instead a 32bit magic, 32bit size, and then the raw bootcode.bin body
<clever>
the sd card code, was meant more as a recovery option, to un-brick things
<clever>
and it was using a file, so you could make the recovery media from a normal pc
<geist>
That Microsoft format, i forget what it was called, used in the rpi nano has a offsets and whatnot in the block itself
<clever>
then the rpi came out, and used the recovery route as the main route
<geist>
Very wasteful, but pretty flexible in the end
<geist>
Since you can scatter something all over the place and reconstruct it later, with checksums on each block
<clever>
nice
<clever>
you mean the pico, and uf2?
<zid>
Don't allow updating the firmware, verify it in the factory, gg ;)
<geist>
Downside is it’s about 2:1 overhead
<geist>
UF2 is yeah i think it
<clever>
yeah
<geist>
Doesn’t have to store it that way on flash but you *could*
<clever>
uf2 uses 512 byte packets, so it nicely sends one packet per MSD write command
<clever>
and then it sends data in blocks of flash write size
<geist>
Yah and you can fake out a FAT partition without actually parsing FAT
<clever>
it could be >512, if your flash chip supports that
<geist>
By just scanning the faked out block device for fragments of the file
<geist>
Since each block is self-identifying
<clever>
but since the pico uses 256 byte flash write blocks, its got the 2:1 waste
<clever>
ive programmed a pico with cat before, cat foo.uf2 > /dev/sda
<geist>
Anyway at first i was kinda grossed out and then started to appreciate the elegance of UF2
<clever>
all it cares about is the uf2 packets being written
<geist>
Yah
<clever>
its abusing MSD as a packet transfer protocol
<geist>
Anyway, gotta go
<clever>
*waves*
<clever>
get out of here!
gog has quit [Quit: bye]
dbana has quit [Quit: Lost terminal]
<clever>
zid: for the roku2, there was per-device keys burned into the OTP
<clever>
zid: and bootcode.bin had to be correctly signed, or it would just not boot
<clever>
the board was capable of reflashing itself in the field to update things, but i can see there being a major bricking risk, if it lacks a fallback
<clever>
pi1/pi0 use the exact same soc, but never enabled that feature
<zid>
The wii had a couple of stages, one of which was fixed inside the cpu itself, and had the keys for the one on flash, way fancier
<clever>
zid: thats pretty much exactly what the rpi has
<clever>
there is a boot rom in the cpu
<clever>
half of the key is in the rom, fixed for the entire batch of chips
gog has joined #osdev
<clever>
half of the key is in OTP, unique to the device
<clever>
the 2 halves xor together, and then hmac-sha1 sign the bootcode.bin file
<zid>
It meant the console was a brick if you never dumped your keys out though and the nand died
<clever>
zid: for the bcm2835, there is a timing exploit in the final signature compare method
<zid>
wii's was better
<zid>
they used strcmp instead of memcmp
<clever>
the more signature bytes you get right in a row, the longer the check takes
<clever>
*facepalm*
<zid>
so you needed 256 attempts to bruteforce a lead zero
<zid>
and you were done
<clever>
for the bcm2711, you just append the signature to the end of the bootcode.bin file
<bslsk05>
github.com: rpi-tools/sign.js at master · librerpi/rpi-tools · GitHub
<zid>
NDS was better, all the hashing worked, but it didn't cover the entry point in the header
<clever>
for the rpi, the entry-point is hard-coded into the rom
<clever>
bootcode.bin gets copied to 0x8000_0000, and if the sig is valid, jump to 0x8000_0200
<clever>
cache-as-ram
<clever>
pi0-pi3 all have the sig checks disabled, so "invalid" and "non-official" files just work, without any fuss
<clever>
roku2 has proper per-device keys, so you need to gain execute first to dump them
<clever>
bcm2711 (pi4/pi400/cm4) has the sig checks half enabled, but every unit has the same keys
<clever>
zid: at some point, broadcom realized what a mistake an hmac was, the bcm2711b1 and further, also have proper RSA signature support, but its not enabled on any boards i know of
<clever>
zid: do you know why an hmac is so bad for security?
gog has quit [Quit: byee]
AssKoala has joined #osdev
freakazoid343 has quit [Ping timeout: 245 seconds]
sts-q has quit [Ping timeout: 240 seconds]
sts-q has joined #osdev
mahmutov has quit [Ping timeout: 240 seconds]
freakazoid333 has joined #osdev
ZipCPU has joined #osdev
srjek has quit [Ping timeout: 240 seconds]
kulernil has joined #osdev
nyah has quit [Quit: leaving]
flx-- has joined #osdev
flx- has quit [Ping timeout: 245 seconds]
justyb11 has quit [Quit: Leaving]
shlomif has joined #osdev
Izem has joined #osdev
<Izem>
what is required to boot a kernel on (sea) bios?
<clever>
Izem: a bootloader of some kind
<zid>
two bytes
<clever>
legacy bios will only load 512 bytes of MBR from disk, and execute it, if it has that 2 byte magic
<Izem>
I know I need a kernel imagine but I don't know what goes between that
<Izem>
s/imagine/image/
<zid>
Try grub, it's easiest
<clever>
simplest answer, use something like grub, and then throw in something like a multiboot header
<Izem>
yeah I am on windows that's why I was wondering if I could just use virtualbox
<Izem>
I suppose grub is available on cygwin
<zid>
i'd just do it in a linux VM but i am way more comfortable there than windows
<clever>
zid: same, i dont know how i would even make a disk image on windows, with tooling i'm familiar with
<Izem>
ok
<fedorafan_altern>
morning
johnjay has quit [Ping timeout: 248 seconds]
johnjay has joined #osdev
paulman has joined #osdev
kulernil has quit [Ping timeout: 244 seconds]
devcpu has quit [Ping timeout: 268 seconds]
devcpu has joined #osdev
superleaf1995 has joined #osdev
Izem has quit [Quit: Connection closed]
mctpyt has quit [Ping timeout: 268 seconds]
rubion has joined #osdev
superleaf1995 has quit [Quit: leaving]
<klange>
thinking about kernel logging suddenly, mostly because I glanced at a printf in my ELF loader and it sparked some further thought
<klange>
toaru32 had 'debug_print' which did a bunch some macro magic, and had different log levels, and would output to nothing, serial, or even certain VFS files (particularly TTYs)
<klange>
misaka has... essentially nothing, there's a few places where code calling debug_print was ported and it's macro'd to nothing, and there's a `printf` that spits to serial / vbox debug log but it's only used in strict debugging situations, and there's an fprintf that can print to userspace TTYs that I've been using in my in-progress modules (ahci, xhci) mostly so I can see the output on my laptop
<klange>
oh there's also a framebuffer bitmap font printer for that, but it's a relic from early bringup of the kernel and its results will happily be wiped away by moving the mouse or the clock ticking
<klange>
I was thinking I need something better for early boot... like, maybe a little reserved area for some log messages, then off to the heap, and then maybe expose that as a character device that something can read in userspace to flush it out to a file? how does Linux do this?
<klange>
I know Linux does some silly thing where log messages are just strings and if you want log levels there's a special low byte at the beginning to say "there's a header here, not just text"?
sortie has joined #osdev
<klange>
it's a sortie
<sham1>
A ring-buffer of messages kinda like dmesg
<sham1>
Well, whatever dmesg reads at least on Linux. It basically has a message log storage buffer thing
<kazinsal>
I should start working on my project again
<kazinsal>
been playing too much FFXIV to do so lately
<klange>
I usually put in a couple of daily roulettes and then crack open the ol' editor.
<kazinsal>
I stopped playing after Heavensward and have spent the last month and a half banging out SB and ShB
<kazinsal>
just finished the 5.0 MSQ last night
<klange>
I can't get the new expansion's benchmark tool to run in Wine, I've tried a half dozen different builds but it's crashing trying to show EULA text (and the config option that suggests it skips that... does not)
<klange>
I've been told there's no actual notable engine changes happening, so hopefully things just keep working fine.
<kazinsal>
yeah, from what I've heard it's effectively the same. I was kind of hoping they'd add some more high end graphics features in for endwalker but apparently not
<kazinsal>
TAA instead of SMAA, etc
<kazinsal>
that being said I get a consistent 120fps even in alliance raid boss pulls on a two-generation-old card so
<sortie>
It's the klange!
<sortie>
ka-lang-gah
<klange>
I usually get a solid 60@1080p but I do experience some dips... when I ran natively in Windows, though, I had no trouble running at 5760x1080 on this card.
<klange>
sortie: how goes the sortixes, I hear your website is self-hosted now? that is exciting!
<kazinsal>
the worst framerate I get on my home world is in the limsa lominsa aetheryte plaza, because of all the reflections and a hundred catgirls (myself included) and a dozen bards playing five different off-tempo renditions of A Cruel Angel's Thesis
<klange>
meanwhile I have neglected my network stack, but somehow my SMP support is stable
<sortie>
It is quite the thing to have GoogleBot crawling my OS
<sortie>
klange, I had soo many worries about things happening if I did this (trouble with my old host, hacking, logs, reliability) but eventually I just went fuck it I'm doing it
<sortie>
I should insert a banner “You may already be a Sortix user.”
<sortie>
klange, wow you did SMP?
<sortie>
ACIPCA or what it's called?
<klange>
Technically, as someone reminded me on the forum, I did SMP _during my lunch break_ (a couple months ago)
<klange>
There's only one ACPI table you need to find the other cores, and it's not got any AML to worry about.
<sortie>
Neat
<klange>
And my bootstrap is this is absolutely awful little straight-to-longmode thing and every step of this has been "I am honestly amazed this works".
<sortie>
Oh wow
<klange>
Like the first time I got secondary cores firing up and scheduling tasks, I got to a functioning GUI before it crashed.
<sortie>
Makes me want to give this a ago
<sortie>
Not for this release cycle (decade) tho
<klange>
Turns out those useless locks I implemented in toaru32 weren't so useless? Though some of them were wrong-but-only-in-the-actually-parallel case.
<sortie>
How much stuff did you do to handle IPIs and such for page invalidation?
<sortie>
I suppose one address space one CPU is a reasonable initial limitation
<sortie>
Aah but the kernel
<klange>
Page invalidation only really matters when you start paging things out, so I've somehow skirted a lot of it - but there is an IPI implemented for it that seems to work.
<klange>
The last couple of bugs were a race condition in my Unix pipes that was mostly presenting as the package manager failing to unpack tarballs (it pipes my gunzip into itself for compressed archives, and it kept getting stuck; many minutes were spent poking at things in gdb)...
<klange>
And a lock ordering issue in my poll-alike, which _appears_ to be the last kernel crash.
<geist>
First time you tear down a thread is probably whenyou’ll start seeing it
<geist>
Since you’ll probably want to unmap the heap, user and kernel
<geist>
And that’s an unmap
<klange>
I think the one place it was clearly necessary for me was unmapping shared memory, but there's just so many places I don't unmap or reuse address ranges that I manage to stupidly avoid the problem.
fedorafan_altern has quit [Remote host closed the connection]
<sham1>
Meanwhile on this old netbook I am in the Linux VTY. No smart quotes for me :(
Arthuria has joined #osdev
kingoffrance has quit [Ping timeout: 245 seconds]
dennis95 has joined #osdev
warlock has quit [Remote host closed the connection]
warlock has joined #osdev
pretty_dumm_guy has joined #osdev
x_ has joined #osdev
x_ is now known as kingoffrance
<sortie>
Sometimes I worry I am quietly judged by iOS people for using '
paulman has quit [Remote host closed the connection]
paulman has joined #osdev
rubion has quit [Ping timeout: 240 seconds]
rubion has joined #osdev
Vercas has quit [Remote host closed the connection]
Vercas has joined #osdev
ElectronApps has joined #osdev
NeoCron has joined #osdev
isaacwoods has joined #osdev
heat has joined #osdev
dormito has joined #osdev
Izem has joined #osdev
rubion has quit [Ping timeout: 240 seconds]
<Izem>
the problem I see with capabilities is that you still have to define which process get's to do what. On the other hand if you have capabilities do you need a micro kernel?
ahalaney has joined #osdev
rubion has joined #osdev
<sham1>
Well, capabilities are indeed very much orthogonal to microkernels, although microkernels might be interested in having capabilities since it's a nice and unified way to make sure that only the programs that are supposed to be drivers have direct access to hardware
<sham1>
If you have a permission/capability system for that, might as well use it for other things as well
<Izem>
I thought in a micro kernel that is getting restricted anyways due to the design?
<sham1>
How could the microkernel tell what process is supposed to be the keyboard driver and what is not?
<sham1>
The kernel still needs to grant the driver access to the hardware. Stuff like IO ports and/or MMIO
<Izem>
ok
<sham1>
Having a "driver capability" of some sort could be used to discriminate this access in such a way that only the driver gets access
<GeDaMo>
Do you mean "it assembles"? Or did you check it produces the required result? :P
<gog>
the former lol
Izem has joined #osdev
<zid>
don't see why it wouldn't
ElectronApps has quit [Remote host closed the connection]
CWiz has quit [Quit: WeeChat 3.2]
<Bitweasil>
"It compiles, ship it!"
<Bitweasil>
:p
<gog>
yes
<kazinsal>
warnings are just reminders that life is fleeting
<GeDaMo>
Warnings are just reminders that you didn't turn off warnings :P
nismbu has quit [Ping timeout: 258 seconds]
<gog>
maybe mov %ax, 2(%rdx, %rbx) ?
<gog>
i like that better than adding in the rip-relative lea
<zid>
depends where the +2 is
<zid>
if it's on the rip GeDaMo's is right
<zid>
pointers are hard
<gog>
well ptr+2(%rbx)
<gog>
is the original
<zid>
right but does "making it rip rel" mean ptr+rip, or rip[all that crap]
<gog>
what i don't understand is adding an offset to a local
<varad>
this did what I wanted : `lea ptr(%rip), %rcx; add %rbx, %rcx; mov %ax, 2(%rcx);`
<sham1>
Warnings are just a reminder of the fact that your code isn't perfect
<gog>
i have no code, therefore all of my code is perfect
<sham1>
nice
<heat>
my ext4 driver got committed with the wrong author so if anything breaks it's technically not my fault!
Arthuria has quit [Ping timeout: 240 seconds]
nismbu has joined #osdev
<gog>
git blame
<heat>
git dont-blame-me
<klange>
ars article on serenity... says they're nearly up to 500 contributors... still doesn't make me feel less shit making comparisons...
<zid>
why do you feel shit
<Izem>
they cited discord in helping the uptake :P
<klange>
partly *because* of that 500 contributor count, I guess... I've been doing this for a decade now and haven't gotten anything close to that kind of attention
<zid>
that's not how fads work and you know it though
<Izem>
that is quite a bit yeah
<Izem>
how would you manage it?
<gog>
step one of emotional self-care: don't compare yourself to others :p
<gog>
(she said unironically)
<Izem>
yeah
<klange>
instructions unclear, stopped comparing myself to baseline health norms and have withered away in my bed
<heat>
how many of those contributors have made actual high quality contributions?
<bslsk05>
github.com: Contributors to SerenityOS/serenity · GitHub
<GeDaMo>
Just get all the voices in your head to start contributing :|
<j`ey>
there's a lot of people with big +-s
<sham1>
klange: I don't know if you've pushed Toaru for publicity as much as Andreas has with Serenity
<sham1>
That's probably a major contributing (heh) factor
<klange>
just makes me want to reject any new contributions to toaru, so i can grasp at the idea that i'm still better because I did this myself... that's what I did with redox after all!
<j`ey>
lol
<klange>
and we have SMP and actual x86-64 support and I'm not an arrogant ass who thinks not having a downloadable pre-made build is somehow a feature
<zid>
do we have nakiri ayame theme pack
<gog>
who?
<zid>
you heard me
<gog>
oh a vtuber
<gog>
i wanna steal her look
<klange>
maybe instead of a PonyOS release, 2.0 stuff should get a VTuber-themed release
<klange>
that's what I really need, none of this usb stuff - gotta get my own TLS implementation so I can compete...
<gog>
thread-local storage or transport-layer security
<sham1>
Why not both
<zid>
transport-local storage
<gog>
thread-layer security
<sham1>
Thread-local security
<heat>
thread-local storage
<klange>
gog: the latter, I have thread-local storage - it was a necessity for supporting threads on my bytecode VM.
<zid>
pirate libressl?
<klange>
For the encrypted stream standard, I've been using mbedtls.
<heat>
argh
<klange>
[formerly PolarSSL]
<zid>
[formerly formerly the software project known as polar ssl]
<heat>
[somehow formerly openssl]
<gog>
lol
<heat>
I'm still traumatised by my openssl port
<heat>
the build system is a big ??????????
<gog>
didn't it have-- yeah i wwas gonna say it had a wacky build system
<sham1>
Speaking of unf-ing build systems, my investigation towards redo seems to pay dividends
<sham1>
Helps that it's actually just shell scripts and thus it can do a lot more than what standard make can do
<Izem>
gnu make can be extended by guile or python if you wish :P
<sham1>
But that's GNU make
<gog>
i use guile for everything
<gog>
just kidding i'm a socially awkward mess hahahahahaha
<gog>
:'(
<Izem>
lol
<Izem>
sham1 "everything" seems to use gnu make
<sham1>
Indeed they seem to, and they probably do. However, I like not relying on implementation-specific stuff if I can avoid it. And besides, I just find the ideas behind redo to be saner than make
<Izem>
what are the major differences?
<heat>
use something that can generate ninja
<heat>
make is shitty
<sham1>
Well as said, redo build files are shell. Also redo is inherently recursive, and avoids a lot of the problems with "recursive make considered harmful"
<sham1>
The way target regeneration is also different since it's not just based around mtime like with make
<heat>
do you need shell scripts all the time to build stuff?
<heat>
that's not a good idea...
<GeDaMo>
gcc *.c -o myprog
<sham1>
Dependencies also don't require hacking around like Makefiles do, if you have gcc -MMD or whatever. In fact, the dependencies can be declared "after" the build according to the information gathered during the build
<heat>
most modern build systems take care of -MMD and whatnot
<sham1>
With 10x the complexity
<heat>
and 10x the usefulness
dennis95 has quit [Ping timeout: 240 seconds]
dennis95 has joined #osdev
<gog>
considering things harmful considered harmful
<heat>
make and ninja are just low level build systems, they're good for the actual backend
<heat>
but if you're a developer and you want to write a build thingy that does what you want and how you want it to, you should go for something higher-level
<sham1>
A lot of these build systems make assumptions about the compilation, whether it's meson, cmake, or whatever. Make, redo and ninja all are just taking a list of dependencies, usually files, and doing stuff with the targets
<heat>
that's why most other build systems output ninja or make
<heat>
what assumptions?
wgrant has quit [Ping timeout: 248 seconds]
<Izem>
doesn't redo use crypto hashes instead of filesystem timestamp?
<Izem>
that's probably a better idea
<sham1>
Yeah. My own implementation uses sha-512 for example, although others may use a combination of things like mtime and the content hash digest
<Izem>
otherwise the idea of make is great
<Izem>
how to make it general and access os features is the trick part
<heat>
why is that a better idea?
<sham1>
Well this is basically recursive make without Makefiles
<sham1>
Because just because your mtime changes, that doesn't mean that you necessarily need a rebuild
<heat>
an mtime check is exponentially faster than calculating a file's checksum
<sham1>
Also things like builds being distributed over multiple machines
<Izem>
it's a tradeoff for sure
<heat>
if you're changing a file and undoing it, well, it's your fault that you're doing nonsense
<Izem>
but I think folks have been happy to have make in their own tools due to the specific tools the need
<Izem>
rake, shake etc
<Izem>
so then you get to what ninja does which is taking instructions and the generation / program is someone else job
<sham1>
Anyway, about assumptions. With things like cmake and meson, you need to teach quite a bit about your system to cross-compile. And "quite a bit" ranges from maybe just creating just a stub file to creating a more substantial file that does stuff
amine has joined #osdev
<heat>
cmake needs a file that's literally, what, 10-20 lines?
<sham1>
That's 10-20 lines more than I want to write
<Izem>
comes with the territory
<Izem>
have you seen auto tools? :P
<heat>
those 10-20 lines save you 200 in other files
freakazoid343 has joined #osdev
<sham1>
Well most targets in programming probably can just be built by one target file, like how you'd use one rule in make
<sham1>
And things like cmake and meson aren't as specific. Meanwhile redo, like make and ninja for example, can be used straight away to build things like LaTeX documents. Of course you can teach cmake and meson those things, but it requires more effort than I'm at least willing to put in
<sham1>
Err, aren't as general
<sham1>
That of course comes from the fact that they're specifically made for programming, but that still feels like a slight limitation
shlomif has quit [Ping timeout: 245 seconds]
isaacwoods has quit [Quit: WeeChat 3.2]
isaacwoods has joined #osdev
mahmutov has joined #osdev
vdamewood has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
tacco has joined #osdev
mctpyt has joined #osdev
varad has quit [Ping timeout: 245 seconds]
Izem has quit [Ping timeout: 252 seconds]
srjek has quit [Ping timeout: 240 seconds]
rubion has quit [Remote host closed the connection]
Izem has joined #osdev
rubion has joined #osdev
<Santurysim>
Hello, do I understand correctly that if I plan to use VESA VBE, I need to do all things that require int 10h in real mode?
<zid>
or emulate your own bios
mctpyt has quit [Ping timeout: 252 seconds]
<geist>
but yeah quickly you find that you can't do much with VBE once you get into protected mode
<geist>
hence why most Real OSes have native drivers for the chipset
mctpyt has joined #osdev
<zid>
you could also use VT-x I guess
nyah has joined #osdev
<zid>
unrestricted guest runs the bios code
<Santurysim>
Why? All I want for now is to display some text
<moon-child>
0xb8000
<Santurysim>
... with bigger screen size
<Santurysim>
I'm working with already existing os, and it has 0xb8000 tty driver already
<bslsk05>
wiki.osdev.org: VESA Video Modes - OSDev Wiki
<zid>
so do that, you asked for alternatives to doing that
<zid>
either you use the bios, emulate the bios, run the bios as a guest in a vm, or write a driver
GeDaMo has quit [Quit: Leaving.]
rubion has quit [Ping timeout: 252 seconds]
mhall has quit [Quit: Connection closed for inactivity]
<sortie>
Santurysim, the easy solution is to use an existing bootloader like GRUB and having it give you the best framebuffer it can arrange
rubion has joined #osdev
<sortie>
GRUB has drivers for virtual machines, VBE, and various things, plus EFI support, so it can do a decent job getting you a framebuffer on a real machine
<sortie>
Of course if you do the use-your-own-bootloader route you're on your own
<sortie>
Personally I went with this route and implemented a simple mode setting graphics driver for common virtual machines
<gog>
it's pretty straightforward with EFI
<gog>
the API already has everything you need
<sortie>
*this route being GRUB
<gog>
(if you're rolling your own)
<sortie>
Yeah so messing with VBE from protected mode usually means you're going down a bad path
<sortie>
Since you can use tech like GRUB or EFI or your own bootloader to solve a basic framebuffer for you
<sortie>
BIOS calls are just massively unsafe from a kernel
<gog>
remember the time before linux had kms and you had to use v86d to use vesa calls in x86_64
<gog>
bad times :p
<gog>
ah yes uvesafb needed it i remember
<gog>
apparently it's still around
<zid>
lynx -g pls
<gog>
yes exactly
<gog>
ok ttg
gog has quit [Quit: bye]
Izem has quit [Ping timeout: 268 seconds]
Izem has joined #osdev
<Izem>
sortie: there's no way to isolate those calls?
<sortie>
Izem, it's worse than that
<Izem>
ah man
<sortie>
Izem, these calls fundamentally program the GPU and whatever hardware specifics
<sortie>
You don't really know what they do
<Izem>
yeah that's tough
<sortie>
Once you're a kernel you're in control of the PCI devices and so on and caching bits and memory layouts etc.
<zid>
I mean, for vbe, they just run the rom supplied by the vga device
<sortie>
Yeah in practice
<Izem>
I guess efi has an upside despite the complexity :P
<zid>
which is why it works for "any" card
<zid>
becuse they all support that interface
<sortie>
But it's hard to know what's going to happen when you do a BIOS call / use VBE, and do so in a safe way
<sortie>
I suppose if you can access that ROM directly, one might be able to use that (mostly?) safely
<zid>
yea who knows if your VBE impl. has a bug that makes it write to a random address nobody cares about from DOS
<zid>
That's why I like the emulation strat
<sortie>
Izem, absolutely EFI has that big advantage
<zid>
Just run it in a sandbox and only do the io port accesses it wants
<vin>
Is there a place where cache latency (cycles) is published per cpu? like wikichip/cpu-world
<Santurysim>
sortie: thank you! As I mentioned, I'm playing with already existing os, and it has its bootloader (however, it is multiboot-capable). Also, it supports bios only at the moment. If I worked on my own os, i would definitely use grub or efi
<sortie>
Santurysim, ah cool, I missed that part :)
<sortie>
Absolutely, you know your constraints :)
air has quit [Ping timeout: 268 seconds]
dormito has quit [Ping timeout: 258 seconds]
<amine>
Hey everyone, I'm following along the xv6 book and came across https://github.com/mit-pdos/xv6-public/blob/master/bootasm.S#L79 in the bootloader code. It seems the global descriptor table is 4 bytes aligned when in the manual and looking around in the internet that's supposed to be 8 bytes aligned. Any idea what I'm missing there ?
<bslsk05>
github.com: xv6-public/bootasm.S at master · mit-pdos/xv6-public · GitHub
<amine>
also the structure of an entry in https://wiki.osdev.org/LGDT shows that the total is 8 bytes (assuming a packed struct) so that made sense to me that the alignement should follow
<bslsk05>
wiki.osdev.org: Global Descriptor Table - OSDev Wiki
<zid>
I think it works with 4 but 16 is prefered for shpeed reasons
<zid>
I could check the manual
<amine>
16 ? now I'm more confused :p
<zid>
The
<zid>
base address of the GDT should be aligned on an eight-byte boundary to yield the best processor performance.
<zid>
8 apparently
<zid>
3.5.1 below figure 3-10
<zid>
(gdt is 16 bytes in long mode so it made sense to me that it'd want 16)
<amine>
yes that's what I was reading
<sham1>
Well the processor can indeed read unaligned data, it's just not that fast
<zid>
I'd align it to 64 cus why not
<sham1>
64 bits or 64 bytes, because those are two very differnet things
<zid>
bytes
<zid>
cache lines are fun
<amine>
yeah but for that link (the xv6-public repo), they set it to 4 bytes which is smaller, doesn't that mean it wouldn't work ?
<sham1>
Future-proofing for the 512-bit CPUs I see
<zid>
no, it says 'for performance'
<zid>
not 'required'
<amine>
hm true. I guess my confusion is once it's set to 4, how is that organized in memory exactly. I just assumed once you have an address say 0x00 you would need 8 contiguous bytes which would fit the struct, but if we set 4 bytes that's almost like it would be truncated as the next address 0x04 would have the next entry.
<amine>
I'm sure Im getting it all wrong :p
<zid>
00 is aligned to EVERYTHING
<zid>
we're talking about alignment, meaning, what the address ends in, think multiplication tables
<zid>
so if your table is at 0xDEADFEE4 then it's aligned to 1 2 and 4
<zid>
if your table is at 0xBEEF0000 then it's aligned to 1 2 4 8 16 32 64 128 256 512 1024 2048 and 4096 :P
<sham1>
Think modular arithmetic
<sham1>
Modular arithmetic in hexadecimal even, because looking at addresses makes it easier to reason about like that
<zid>
0x00007 is aligned to.. 7, stupid primes
<sham1>
And it of course happens that for powers of two specifically, there is a very neat way of testing if something is divisible by said power
<sham1>
Let x be 2^n where n is some positive integer (could also be zero, but that wouldn't be interesting)
<sham1>
Now, addr mod x can be represented as addr & (x - 1), that is, you grab the n least significant bits of the addr and check if they're zero. If they're all zero, it's divisible
lg has quit [Ping timeout: 258 seconds]
<sham1>
And of course, because this is such a well-known optimization, compilers will turn code like `num % 16` into bitwise ANDing num with 15, or 16-1
<sham1>
On that note, I do slightly cringe when people do something like (num & 1) to check if something is divisible by two, for this very reason. Your compiler is smart. Use it
<zid>
num & 1 is just as idiomatic to me
lg has joined #osdev
<amine>
yeah I think I get that now. Here is my confusion: say I have a `foo` array of a packed struct of 64 bits ( the gdt entries), I assumed that by declaring it 4 bytes aligned, the memory addresses would be chosen in a way that it's exactly 4 bytes between each entry contiguously. So | entry1 (4bytes)| entry 2 (4bytes)|...| entry N | where address of
<amine>
entry1 could be 0x0, entry2 0x4, entry3 0x8, etc .. in that exact order. But I think I get it now, that's not necessarily the case and it's just about what the address would end up with, the rest would be figured out automatically in terms of where exactly each element is ?
wgrant has joined #osdev
<sham1>
Well the GDT entries should just fit snugly next to each other without any padding. Indeed, that's what the CPU requires and why you have to use packed structs if you want to represent the GDT in C for some reason
<zid>
or use chars :P
<amine>
right so choosing 4 bytes alignement or 8 would essentially have the same result in terms of how things are in memory right ?
<zid>
but no, that alignment directive is just a directive to align the current output location to n
<zid>
so if you were on byte 0x123 of your output and you told it to align to 8, it'd output 5 bytes of padding to get you to 0x128
<zid>
then your data follows, completely unrelated
<sham1>
Yeah. The initial address of the array just has the alignment. The elements then get their natural alignment which in this case is ignored in favour of being packed tightly
<amine>
ohhh
<amine>
that makes sense
<zid>
which has the effect of.. aligning your data
<amine>
thanks zid and sham1 :-)
<zid>
it doesn't change the padding in your struct, just changes its alignment
<zid>
your 37 bytes are still 37 bytes, regardless of whether they start with an address ending in a 4 or a 7
<sham1>
But yeah. chars or uint64_t. After all, GDTs are specific to x86 and AMD64 so the undefined behaviour of potentially reading unaligned data should still work
<zid>
idb
srjek has joined #osdev
<heat>
Izem, EFI's GOP is still running a rom supplied by the GPU
<heat>
there's no advantage
<heat>
it just supports more stuff, but that's about it
<Izem>
ok
ahalaney has quit [Quit: Leaving]
dennis95 has quit [Quit: Leaving]
heat has quit [Remote host closed the connection]
Izem has quit [Quit: Connection closed]
mahmutov has quit [Ping timeout: 248 seconds]
mahmutov has joined #osdev
gog has joined #osdev
NeoCron has quit [Remote host closed the connection]
AssKoala has quit [Ping timeout: 258 seconds]
isaacwoods has quit [Quit: WeeChat 3.2]
nismbu has quit [Ping timeout: 252 seconds]
rubion has quit [Ping timeout: 258 seconds]
sortie has quit [Quit: Leaving]
mahmutov_ has joined #osdev
mahmutov has quit [Ping timeout: 252 seconds]
mahmutov_ has quit [Ping timeout: 248 seconds]
freakazoid343 has quit [Read error: Connection reset by peer]