<bslsk05>
twitter: <glowcoil> The x86 ISA is a declarative format for specifying parallel compute tasks. Units of work are scheduled dynamically across a set of compute and storage resources at runtime, with dataflow dependencies specified using a finite set of labels called "logical registers"
<clever>
lol
gorgonical has quit [Quit: ZNC 1.6.5+deb1+deb9u2 - http://znc.in]
gorgonical has joined #osdev
heat has joined #osdev
<heat>
does qemu, or any vm for that matter, implement acpi suspend/resume?
<clever>
heat: closest answer i can give, is that if darwin tries to suspend the vm, it hangs in qemu
<clever>
so either darwin is doing it weirdly, or qemu isnt implementing it properly
<bslsk05>
www.nasm.us: NASM - The Netwide Assembler
pg12 has quit [Ping timeout: 256 seconds]
<Clockface>
thankds
<clever>
geist: that reminds me, are you working on LK's fat support? it was broken last i tried to us eit
<geist>
yes.
<clever>
nice
<geist>
it was very broken
<clever>
i had an alternative fat library i could have ported, but duplicating fat support didnt seem wise
pg12 has joined #osdev
<clever>
thats part of why i worked on ext2/3/4 as well, to have anything
<clever>
oh, and will your fat be ro or rw?
<clever>
write support is something i also need to work on
<geist>
rw
<clever>
nice
<geist>
just got readdir working, proper LFN support, etc
<geist>
ot
<clever>
so that just leaves sdhost write support on my plate
<geist>
it's almost fully RO as of a few minutes ago
<clever>
i was also talking to somebody else that was testing their own fs code, and they just compiled the module to linux userland
<clever>
a few shim functions, and you could easily interface LK's fs modules with linux userland
<clever>
hardest part might be getting LK's build system to co-operate, or just ignoring it entirely
<geist>
am having a good utilization of lightweight C++ too here
<geist>
it is definitely coming in handy
<clever>
yeah, basic c++ features like templates and constexpr have zero runtime cost, and feel a lot more type-safe then macros
<clever>
atleast how i'm using them, for a single inline asm in a templated function
Likorn has joined #osdev
<clever>
if (rep == 1) { // no repitition
<clever>
template <typename T> static inline void vst(int x, int y, T *dst, bool xinc, bool yinc, int rep, bool horizontal, int stride) {
<moon-child>
c++ templates are duck typed :P
<clever>
this will basically accept any type T, and then can sizeof it and do the right thing
<clever>
ideally, it would only accept 8/16/32bit ints, but i dont know how to specify that
<geist>
yah built a nice like directory walker state object that really works nicely
<geist>
walking through the DIR entries are annoying in FAT because they can cross sector boundaries so you have to parse it an entry (0x20 bytes) at a time
<geist>
but it's nice to have a helper iterator object that you just pass into deeper routines and it automatically pushes the sector forward. it came out nicely
<clever>
can a single 0x20 byte entry cross a sector? or is that not possible (math says exactly 16 entries per 512 byte sector)
<geist>
at least that's not the case, but long file name entries are a series of 0x20 entries
<geist>
and those can cross sector boundaries
<clever>
ah
<clever>
so you could just have a simple fetch_dir_entry(dirhandle, index)
<geist>
so you have to parse them basically 13 characters at a time. at least they're guaranteed to show up in a certain order
<clever>
divide index by entries_per_cluster, check the fat table, oh right, fat is singly linked
<clever>
seeking gets costly, the longer the object is
<geist>
yah. also i have it hold a pointer to the block cache open as it walks forward, so it's prettyu efficient
<clever>
yeah, i think i saw similar in the ext2 code, where it only re-reads the block when crossing over
<clever>
ext is complicated by records being variable length, but still cant cross a block
<geist>
yah
<clever>
so there is a padding record to finish it off
<geist>
yah more or less the asme thing here
<geist>
since you have to accumulate the entries as yuo cross sector boundaries, but it's built such that they show up in a particular order, so it's not too bad
<clever>
how do the LFN entries survive, if a non-LFN os is creating new files?
<clever>
wont it compact the dir listing, or reuse the first "deleted" entry it can find?
Likorn has quit [Quit: WeeChat 3.4.1]
<geist>
two mechanisms: the LFN entries appear in reverse order, with a sequence number in them and a marker for the last one
<geist>
so it starts off with 'LFN entry 4, END sequence' then LFN3 LFN2 LFN1
<geist>
so that helps at least pick out a corruption
<geist>
the cheezy and not very good hack is it stores an 8 bit checksum of the SFN its paired with
<geist>
so to be more correct a long LFN looks like LFN4 LFN3 LFN2 LFN1 SFN
<clever>
but wont the entire LFN be lost, if another os creates a regular SFN in one of the LFN slots?
<geist>
ie, it must always be followed by a legacy SFN
<geist>
right
<geist>
and the checksum stored in the LFN entries is basically of the SFN entry
<geist>
so it attempts to make sure if the SFN is changed, the LFN should fail to match it
<clever>
i also heard that modern windows ntfs, isnt generating SFN's by default now
<clever>
and thats caused some problems at my work before
<geist>
and if a LFN is trashed, the checksums wont match bertween the sequences
<geist>
yah NTFS does it more nicely in that LFNs and SFNs are simply two different name attribute types associated with the file
<geist>
so it's easy to simply leave out one of them
<geist>
i do forget how it works in dir entries, but the FILE record itself has a copy of all the names a file appears under, including SFN and LFNs
<clever>
in my case, it was a database library using a non-unicode capable CreateFile (the windows version of open)
<clever>
and a large chunk of the userbase have asian characters in their windows username
j00ru_ has quit [Ping timeout: 248 seconds]
<geist>
that's a downside and i think a misdesign of NTFS: the file record (akin to a unix inode) stores a copy of the name of the file
<clever>
so you either cd and use relative paths, or you convert to SFN and open that
<clever>
and with the SFN missing on files...
<geist>
so if you rename a file you both modify the dir entry in the dir btree *and* the FILE record
<clever>
that sounds weird
<geist>
which works because hard links are generally not a thing you use much on NTFS, though there is an ability to do that
<clever>
yeah
<geist>
yah i thnk that's amisdesign personally, but it does make the DIR entries kinda elegant: if you consider a FILE record to be just a list of sorted attributes (which is what they are), the NAME attribute is ust one of them
<geist>
and a directory file is simply a btree index of the name attribute
<geist>
so it's kinda an elegant design
<clever>
ext's btree stuff is a bit of a hack
<clever>
in the same vein as fat LFN
<geist>
yah i can bet. i haven't looked into that yet
<clever>
the btree is hidden within abnormally long and unused directory entries
<geist>
probably will be a disaster to implement
<geist>
anyway just pushed the last bit of FAT code
<geist>
it's in pretty good shape now, for RO. need to fix the FAT12 support
<bslsk05>
ext4.wiki.kernel.org: Ext4 Disk Layout - Ext4
<clever>
the old linear directory format, was just a 32bit inode, 16bit record length, 16bit name length, the raw name, and a hard limit of 255 bytes for a filename (263 bytes for the whole record)
<clever>
the new linear format, is a 32bit inode, a 16bit record length, an 8bit name length, and an 8bit type field (so readdir can give a type without loading the inode), then the raw filename
<clever>
dir leaf blocks can also have a checksum, the final record has inode=0, namelength=0, recordlength=12, and a checksum of that one block in the name field
* geist
nods
<geist>
only enough brain to hold one FS in my head at a time
<clever>
and the hash tree root is stored in the 1st data block of the directory
<clever>
using an inode of 0 again, to look like a deleted file
<clever>
switching gears a bit, i was talking on the osdev discord yesterday, about how caching works in EL1, when running under an EL2 hypervisor
<clever>
from a skim over the arm arm, it looks like EL2 can forcibly enable the d-cache, and the EL2's extra paging tables control what is cacheable?
<bslsk05>
www.theregister.com: Samsung, others test Esperanto's 1,000-core RISC-V AI chip • The Register
elastic_dog has joined #osdev
vinleod is now known as vdamewood
Likorn has joined #osdev
elastic_dog has quit [Ping timeout: 250 seconds]
<zid>
"crappy jellybean isa actually useful after you strap 1100 of them together"
<zid>
They should hire me
elastic_dog has joined #osdev
divine has quit [Ping timeout: 248 seconds]
divine has joined #osdev
Burgundy has joined #osdev
Likorn has quit [Quit: WeeChat 3.4.1]
vdamewood has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
dennis95 has joined #osdev
nyah has joined #osdev
Vercas4 has joined #osdev
Vercas has quit [Ping timeout: 240 seconds]
Vercas4 is now known as Vercas
freakazoid12345 has quit [Ping timeout: 240 seconds]
gog has joined #osdev
pretty_dumm_guy has joined #osdev
gildasio has quit [Ping timeout: 240 seconds]
gildasio has joined #osdev
Viy has joined #osdev
<Viy>
Hello! Is anyone here good with QEMU and/or Raspberry Pi? I can't get the ATAGS. 0x100 is empty and r2 is set to zero.
<j`ey>
shouldnt you use DT instead?
<Viy>
I thought that was for ARM64 and I'm trying to use ARM32, but if I can get that how would I find it?
<j`ey>
32 bit also supports DT
<j`ey>
hm, r2 should be the DT address.. so if its 0 that doesn sount good
<Viy>
From the little I could find from Google, Qemu will only initialize the registers in certain situations, but beyond using an ELF file (which I am) I can't find any more details.
<Viy>
And I stepped through the instructions in GDB so I know it's not that I overwrote them.
Burgundy has quit [Ping timeout: 272 seconds]
heat has joined #osdev
<heat>
is the standard 400ns stall for IDE controllers not enough?
heat has quit [Remote host closed the connection]
heat has joined #osdev
wootehfoot has joined #osdev
ping- has quit [Ping timeout: 252 seconds]
dennis95 has quit [Quit: Leaving]
Ermine has quit [Quit: Cya!]
Ermine has joined #osdev
freakazoid333 has joined #osdev
Haffi_ has joined #osdev
xenos1984 has quit [Remote host closed the connection]
xenos1984 has joined #osdev
<heat>
ok, weird shit
<heat>
i write 0 bytes to the drive but virtualbox zeroes it out somehow
Haffi_ has left #osdev [#osdev]
Likorn has joined #osdev
wootehfoot has quit [Read error: Connection reset by peer]
dude12312414 has joined #osdev
<heat>
WTF
<heat>
THIS IS VIRTUALBOX BUG
<heat>
A*
<heat>
I do "qemu-img convert -f raw -Oqcow hdd.img hdd.qcow" to get a qcow image, I pass it to virtualbox, and it works for the first run, then everything gets zeroed
<heat>
just tested it with alpine linux, same thing
<heat>
I can boot, mount it, read the filesystem, but as soon as I reboot, it's dead
gmodena has joined #osdev
<Ermine>
sata?
<heat>
yes, IDE
<heat>
it seems that qcow support is busted
<heat>
yupp, for sure
<Viy>
Well then you just make VirtualBox crash when you want to exit.
<heat>
exit(0) in virtualbox considered harmful
sebonirc has joined #osdev
Likorn has quit [Quit: WeeChat 3.4.1]
<Ermine>
generate triple fault?
gxt has quit [Remote host closed the connection]
gxt has joined #osdev
Viy has quit [Ping timeout: 256 seconds]
j00ru has joined #osdev
<geist>
heat: is there a reason you're not using virtualbox's native vdi? is it because you want to switch between it and qemu?
<geist>
also maybe try qcow2? I think the 2nd version is much more powerful/supported/modern
<geist>
though i dunno if v ox supports it
nyah_ has joined #osdev
nyah has quit [Ping timeout: 272 seconds]
Celelibi has quit [Ping timeout: 240 seconds]
j00ru has quit [Ping timeout: 276 seconds]
j00ru has joined #osdev
<heat>
geist, just wanted a quick format I could test vbox with
<geist>
my take is qcow 1 may be too old, try qcow2 or vdi, the latter of which is native to virtualbox
<heat>
first one I though of was qcow (which is supported by vbox)
<geist>
and i think the qemu-img too may support?
<geist>
qcow or qcow2?
<heat>
idk if vdi is supported by qemu-img
<heat>
qcow
<heat>
apparently in 2019 qcow2 still wasn't supported (and I think it still doesn't, looking at the file extensions it looks for in the selection dialog)
<heat>
anyhow I switched to vmdk and it totally fixed everything
* heat
is annoyed that they don't support raw images
<geist>
i think they do, but it's via vdi
<geist>
basically there's a way to build a raw vdi. i've done it in the past manually
<geist>
basicallyt you end up with a .vdi file (which is some xml/json/etc looking thing) that just points at a raw image
<geist>
or even a /dev node
<geist>
i dont have it handy but you can do it pretty easily
<heat>
huh
<heat>
that would be kinda nice I guess
<heat>
I usually like the raw images better since they're way easier to mount in linux
<geist>
agreed
<geist>
ou can also do `VBoxManage convertfromraw` it seems, but i think that makes a copy
<geist>
but apparently will sparsify the result, etc
<heat>
sparse works fine, i actually prefer it
<geist>
indeed.i was creating 2TB FAT32 images yesterday for testing. sparse FTW!
Celelibi has joined #osdev
<heat>
:D
heat has quit [Remote host closed the connection]
heat_ has joined #osdev
<geist>
was pleased to find that mkfs.fat on linux actually lets you create disk image files based on how many clusters you want. it skips a step of trying to up front compute it to create different disk images
<bslsk05>
github.com: lk/mkblk at master · littlekernel/lk · GitHub
<geist>
clusters are the units of file block in FAT. always a power2 multiple of sector
<heat_>
ah right block sizes
<geist>
so you might have 512 bytes sectors but 4K cluster sizes
<heat_>
mkfs.ext2/3/4 also has that
<geist>
yep. it's nice
<geist>
it also properly creates a sparse file for you
<geist>
not that it's difficult to do so otherwise, but it's a nice one step thing
heat_ is now known as heat
<heat>
geist: offtopic but do you know if there's a way to check if there's something behind a 16550 serial port? or do you just need to timeout?
<geist>
aside from hoping it uses DTD and whatnot lines, not that i know of
<heat>
fuchsia seems to return ZX_ERR_SHOULD_WAIT back to the write() caller
<geist>
i think that's why windows has historically probed serial ports whe it boots, looking for various things
<geist>
the full rs232 spec has all these addtional control lines you can use to exactly detect that the thing on the other side is there and ready to talk but most of that doesn't get used nowadays
<heat>
i noticed my OS was totally halting when I was getty'ing the various tty's because vbox didn't attach anything to serial
<geist>
and that's in addition to the usual RTS/CTS flow control lines
<geist>
there's DCD and DTR and whatnot
<heat>
I was just doing while (!tx_empty()){} under a spinlock and that just hanged my system
<heat>
since nothing's consuming the tx
<geist>
do you have hardware flow control enabled?
<heat>
no?
<geist>
i dunno!
<geist>
this is a driver you wrote in your OS to talk to 16550?
<geist>
is this on qemu
<geist>
?
<heat>
no I don't
srjek has joined #osdev
<heat>
yes for the first, no to the second
<heat>
this works fine for qemu, that's why I didn't catch it
<geist>
TX shoudl normally just consume it without flow control, if TX is enabled
<geist>
unless theres literally no serial port there
<geist>
maybe that's what you mean by 'nothing is behind it?'
<geist>
i assumed you meant there's a serial port there, but nothing is connected on the other end. that's hard to detect. if you mean you dont know if you're actually talking to a serial port, i think there are a few tricks to detect it
<geist>
but the official solution is to read ACPI
<heat>
when disconnected or if it's trying to pipe stuff to a file/socket that is closed
<geist>
oh. is virtualbox not consuming your TX? fuck if i know then
<geist>
that doesn't seem legal
<heat>
yup basically
<geist>
a real serial port would just consume data, unless you have hard flow control enabled and the 'other side' is holding you off
<geist>
but if you have explicitly not enabled it then i think thats virtualbox being nonstandard
<geist>
sounds like effectively they're enabling a form of virtualized HW flow control even if you're not asking for it
<heat>
i can't remember the details of this thing
<heat>
just trying to get my OS to work on vbox since it's the standard virtual machine for windows people
GeDaMo has quit [Remote host closed the connection]
<heat>
haven't touched the serial drivers in some time
* geist
nods
<geist>
serial drivers are surprisingly tricky. it's hard to convey this to folks that are first time fiddling with it
<geist>
and virtualized environments are like an exponential factor of complexity, since they frequently behave not like real hardware
<geist>
and real hardware doesn't have all the bits to properly deal with it
<geist>
vs like something like a packet or block interface, which maps into the virtual world much more directly
<gog>
helo
* geist
pets gog
<geist>
helo!
<geist>
sun is out finally, the kitty has found the sunbeam
<gog>
same
<gog>
nice evening for a dog walk
<gog>
very agreeable springtime weather
vancz has quit []
pie_ has quit []
vancz has joined #osdev
pie_ has joined #osdev
<heat>
guess i'll just add a timeout
<heat>
seems easier
<mrvn>
serial on qemu just write to the FD on the host and when that blocks the virtual serial blocks.
<mrvn>
I think it ignored everything but the irq/fifo control bits
<clever>
one minor detail i found in LK's source, is that if the guest stops reading the rx fifo, then i think qemu will stop reading the host FD
<heat>
virtualbox looks really thorough in certain parts of the emulation
<clever>
and things back up properly
<clever>
and LK will turn the rx irq off, when the software fifo is full
<mrvn>
clever: have to. otherwise it just triggers again. It's level triggered I believe.
<clever>
yep
Likorn has joined #osdev
Burgundy has joined #osdev
<geist>
also if you're TX driven uart this problem would sort itself out too: you fill the fifo, then queue the rest
<geist>
and the TXirq never fires
<geist>
so you dont end up spinning forever. that's possibly how LK avoids it in this situation since the x86 uart should be fully IRQ driven
<geist>
s/TX driven uart/IRQ driven TX/
<heat>
guess that's how linux does it
<heat>
the tty probably queues it like that
<clever>
ive been thinking about how i would use the rpi dma to accel things lately
<geist>
yah probably wy virtualbox Just Works for lots of stuff
<geist>
does it boot linux? ship it
<clever>
but that only helps for things over a certain size
<heat>
linux somehow knows that some ports don't even exist though
<heat>
even though they're on the ACPI tables
<heat>
my detection can't
<heat>
looking into the monolith that is the uart8250 in linux is pretty hard and not something I want to do xD
dude12312414 has quit [Quit: THE RAM IS TOO DAMN HIGH]
<heat>
i should write an nvme driver
<heat>
it's allegedly simple
jhagborg has joined #osdev
archpc6 has joined #osdev
archpc has quit [Ping timeout: 276 seconds]
archpc6 is now known as archpc
elastic_dog has quit [Ping timeout: 248 seconds]
elastic_dog has joined #osdev
gog has quit [Ping timeout: 276 seconds]
gog has joined #osdev
jhagborg has quit [Ping timeout: 272 seconds]
<Clockface>
if you need to need something to be global, and it will be compiled into other peices of software, you can give it a name that no sane person would use to make sure nothing will collide with it
<clever>
Clockface: i think when linking a dynamic library, you can restrict what symbols are exported
<clever>
so symbols a module exports, arent fully exported
heat has quit [Read error: Connection reset by peer]
heat_ has joined #osdev
<Clockface>
i usually do this when im jamming my source code together
<Clockface>
its worked out ok so far since i can write stuff down and copy-paste it
<Clockface>
probably bad to do for some reason, but its been fine for me
<Clockface>
if someone else has to deal with it, i guess then its disadvantages become apparent
<clever>
oh right, and c++ namespaces entirely solve the problem
<clever>
behind the scenes, its just prepending the namespace to all symbols
<Clockface>
sometimes thats how i name them manually