<heat>
sadly there are no open docs but I guess that's ok
<heat>
nobody will actually run anything other than linux on a phone
<heat>
(and maybe uboot)
<mrvn>
can one use clang-tidy with gcc?
<heat>
yes
<heat>
clang-tidy is a separate tool
<heat>
you just have to give it the flags you used
attaboy has quit [Ping timeout: 256 seconds]
troseman has joined #osdev
kkd has quit [Ping timeout: 240 seconds]
cyberbanjo has quit [Remote host closed the connection]
kkd has joined #osdev
Clockface has joined #osdev
vdamewood has quit [Read error: Connection reset by peer]
vdamewood has joined #osdev
<mrvn>
Remeber yesterday I complained about https://godbolt.org/z/a7cTo6hnE not optimising the throw/catch away? There is a paper from 2018 about rethinging the exception mechanism using std::expected. https://www.youtube.com/watch?v=GC4cp4U2f2E&t=2850s for an example and the paper right after. Interesting to watch the whole talk.
<bslsk05>
godbolt.org: Compiler Explorer
<bslsk05>
'CppCon 2018: Brand & Nash “What Could Possibly Go Wrong?: A Tale of Expectations and Exceptions”' by CppCon (01:02:41)
<mrvn>
and with that I bid your good night.
sonny has joined #osdev
phr3ak has quit [Ping timeout: 256 seconds]
phr3ak has joined #osdev
<geist>
hmm, i wonder if there was ever an extention to tag case statements for priority
<heat>
[[likely]] and [[unlikely]] apply to cases
<geist>
huh! I was jsut lookingat __attribute__((hot)) and cold too
<geist>
ah looks like [[likely]] is exactly the same as hot
<heat>
weird behavior
<geist>
because if you put likely and cold at the same time it says cold conflicts with hot
<heat>
if I have two hots with a cold it hots the cold and colds the hots
<geist>
ICY HOT
<heat>
it starts jumping to the hot statements
<heat>
if you remove case 2's hot, it does the sanish thing
<heat>
this thing doesn't work with gcc < 9 and clang
<heat>
"Hot/cold partitioning is now more precise and aggressive."
<CoffeeMuffin>
For non-functions it has only consequences on optimization.
<CoffeeMuffin>
For functions it does a lot more than that
<friedy10->
I thought switch statements are just lookup tables. Why would the priority thing make a difference?
<heat>
switch statements are not just lookup tables
<heat>
CoffeeMuffin, what does it do besides putting the function in a .text.hot/.cold?
<friedy10->
Isn't just a jump table. It's not like you are doing a lot of comparisons like if/else right?
<moon-child>
it might
<moon-child>
jump table only makes sense if you're dense, have many entries, and are unpredictable
<clever>
moon-child: the rpi firmware has a very sparse switch-case in one part of its code, and it compiled into a giant rats-nest of jump tables and if statements
<friedy10->
So jump tables are just a compiler optimization then?
<heat>
yes
<moon-child>
I wouldn't put it thus
<clever>
the compiler found each dense sub-range, and then used if statements to select which jump-table to use
<moon-child>
jump tables are one strategy a compiler might use to compile switch statements
<heat>
from this same gcc 9 change log: " Switch expansion has been improved by using a different strategy (jump table, bit test, decision tree) for a subset of switch cases. "
<heat>
so it has at least three strategies
<CoffeeMuffin>
heat, it optimizes code locality according to branch predicition.
<clever>
the rpi VPU also has a dedicated opcode just for switch jump tables, you basically just do `switch r0`, and then immediately following that opcode, is an int16_t[n] table of pc-relative offsets
<clever>
so the cpu will jmp (pc + (table[r0]<<1))
<clever>
where table itself, is just pc+2 i think
<clever>
and there is an int8_t[n] variant, for when all offsets are small
sonny has quit [Ping timeout: 256 seconds]
nyah has quit [Ping timeout: 272 seconds]
Ameisen has quit [Quit: Quitting]
Ameisen has joined #osdev
cyberbanjo has joined #osdev
sonny has joined #osdev
sonny has left #osdev [#osdev]
heat has quit [Ping timeout: 250 seconds]
gog has quit [Remote host closed the connection]
gog has joined #osdev
[_] has joined #osdev
[itchyjunk] is now known as Guest4684
[_] is now known as [itchyjunk]
Guest4684 has quit [Ping timeout: 240 seconds]
h4zel has joined #osdev
gog has quit [Ping timeout: 272 seconds]
srjek has quit [Ping timeout: 240 seconds]
kingoffrance has quit [Ping timeout: 240 seconds]
<kazinsal>
gah! courier didn't bother trying to call me or anything when attempting to deliver my vax so now I have to go pick it up from a shipping facility tomorrow.
<kazinsal>
lazy gits
<kazinsal>
oh well, my console cable is still in purgatory so I can't use the thing anyways
kingoffrance has joined #osdev
h4zel has quit [Ping timeout: 272 seconds]
h4zel has joined #osdev
h4zel has quit [Ping timeout: 240 seconds]
h4zel has joined #osdev
dude12312414 has joined #osdev
dude12312414 has quit [Client Quit]
h4zel has quit [Ping timeout: 272 seconds]
Jari-- has joined #osdev
troseman has quit [Ping timeout: 272 seconds]
h4zel has joined #osdev
ElectronApps has joined #osdev
k8yun has joined #osdev
attaboy has joined #osdev
archpc has quit [Ping timeout: 240 seconds]
archpc has joined #osdev
eddof13 has joined #osdev
archpc5 has joined #osdev
archpc has quit [Ping timeout: 272 seconds]
archpc5 is now known as archpc
cyberbanjo has quit [Remote host closed the connection]
Electron has joined #osdev
ElectronApps has quit [Ping timeout: 272 seconds]
eddof13 has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<junon>
Long time no see :) Hope everyone's been doing okay
<junon>
The war has had me busy, been helping bellingcat with their investigation. Not a lot of time to work on osdev until today.
<junon>
Potentially dumb question: is there a benefit to allocating contiguous pages where possible during a multi-page allocation for a virtual address space?
<Mutabah>
Not really
<Mutabah>
You're better served by doing cache-aware allocations
<junon>
What does cache-aware allocation mean in this case?
xenos1984 has quit [Remote host closed the connection]
<Mutabah>
The CPU's cache is often keyed off semi-random bits in the address, so a cache-aware allocator would try to avoid allcating nearby pages using the same cache "colours"
<Mutabah>
But really, just do random allocations and it'd work out
xenos1984 has joined #osdev
<junon>
Wouldn't a cache aware allocator have to have intimate knowledge of the specific CPU it's targeting? Not just the ISA but the actual model etc.?
<junon>
It sounds hard to generalize, no?
<junon>
Just making sure I understand
<Mutabah>
Probably... I've never written one
<Mutabah>
Sorry, probably shouldn't have mentioned it
<Mutabah>
iirc the name for that is "cache colouring"
<junon>
Cool thanks, that's helpful!
<moon-child>
I thought cache is usually keyed by the low (intra-page) bits?
<moon-child>
and l2/l3 doesn't matter quite so much
<moon-child>
also doesn't vipt obviate that anyway?
<geist>
i *think* it doesn't matter that much anymore, but at some point when caches were much more direct mapped and maybe only one or two levels i think it was more important
<junon>
Reading more on some of this, I see that Linux has special functions for obtaining pages within a "DMA zone", which is a new concept to me. Is that like a region of physical memory that is capable of being written to by hardware directly?
<geist>
yah based on limitatinos of hardware
<geist>
ie, hardware that can only access 0 - 16MB or 0 - 4GB
<geist>
therefore linux has to carve up pages into various zones
<junon>
Is that specified by the ISA? Or is it indicated in e.g. ACPI?
<Mutabah>
and some hardware doesn't support scatter-gather DMA, so requires contigious addresses
<geist>
junon: it's specified by general knowledge of the capabilities of that particular device, unforutunately
<geist>
it's usually a device limitation, or a limitation of the bus the device is attached to
<geist>
and it's generally Simply Known
<geist>
ie, a 32bit PCI card in a machine with > 4GB ram
<geist>
the card itself cannot address > 4GB so any DMA you tell it to do has to be from a lower address
<junon>
But how does the PFA know to carve out certain areas of physical memory at runtime?
<geist>
generally hard code
<junon>
oh really, wow
<geist>
you simply know that on this sorto f computer (ie, a PC) that this sort of thing exists, so you carve it out ahead of time
<geist>
with the idea that you'll probably need it
<geist>
i actually dunno if there's a command line switch or whatnot on linux on x86-64 to tell it to not create these zones
<junon>
so it depends on the ISA + the hardware detected at boot? Or do they generalize it per cpu type or something and just hope that it's correct?
<geist>
not ISA. the machine
<geist>
ie, a PC, or a raspberry pi or watnot. it's a function of the hardware, not so much the ISA of the computer
<junon>
Oh so that's why Linux has machine-specific builds
<geist>
how it knows to do this, dunno. possibly simply a boolean that defaults to false or true at compile time
<geist>
and possibly overridden by something early in boot time
<junon>
I see
<geist>
my suspicion is if there's any possibliyt of this sort of hting you simply do it up front. so probably ost things that have an x86-64 cpu and say have a arm64 cpu
<geist>
it's not so much isa specific as its hardware that is commonly using that ISA
<geist>
i'm being a bit pedantic, but to me ISA is == the instruction set, nothing more
<geist>
the architecture of the cpu is the larger set of things, and the machine that you build around it is even more
<junon>
Yeah sorry, you're right, I'm using ISA too liberally haha
<geist>
in my code i tend to think of it as arch, platform, target in that order
<junon>
Is DMA something I should factor into the design early on? Or is it something that can be treated as an afterthought?
<geist>
ie, ARCH = arm, platform = broadcomm cpus that go in raspberry pi, target = a raspberry pi
<geist>
on a PC it's kinda an exception: arch = x86, platform = the pc platform
<geist>
target == nothing really
<junon>
I see
<geist>
linux annoyingly fips this around and puts platform specific stuff in their arch dir
<geist>
which is a mistake IMO. the bsds dont do this
<junon>
Yes, that is something has confused me in the past for sure.
<geist>
but. it is what it is. linux i think does it based on the observation/laziness that arch is *usually* associated with a platform
<geist>
and that platforms *usually* dont have more than one arch
<geist>
therefore you can stuff the platform logic in the arch dir. which generally works, except where it dosent
<junon>
Is DMA something that is opt-in? Or is it required to use certain hardware?
<geist>
yes and yes
<geist>
depends. some hardware you can not use dma. most modern hardware *requires* it
<junon>
Gotcha
<geist>
most modern hardware is simply designed around the notion that the hardware itself also can go read/write memory, and that is 'bus mastering dma'
<geist>
early on that was exotic, but now it is simply The Way
<geist>
but you can see the issue: if you had say a rtl8139 ethernet card, which can dma when sending and receiving packets, bbu only has 32bits of addressing
<geist>
then if you put it on a machine with 8GB ram, you can't tell it to DMA to 'high memory'
<geist>
should you worry about it? probaby not. depends on how old hardware you want to support
<geist>
modern stuff (ie, Pc hardware in the last 10-15 years) will almost certainly be 64bit capabile
<geist>
so you can probably safely ignore this problem
<mrvn>
I wouldn't worry about it. By the time your OS is at a point where it is relevant all the old stuff that still needs it has died out. Unless you specifically want to build a kernel for legacy systems for fun.
<geist>
or, if you have 32bit only hardware if you say grabbed all the memory it'll need up front early on in the boot process, you ca probably have a slow but successful allocator that grabs a run of low memory and then just holds onto it for the duration of the boot
<geist>
that's basically what we do in zircon for now and have avoided the problem thus far
[itchyjunk] has quit [Read error: Connection reset by peer]
<mrvn>
Worst case you can always go through all the processes and swap out physical pages till you have the right ones free. Slow but you never ever need it, on average. Loading a driver module that needs special memory is a once in a boot kind of thing.
<geist>
right, even linux's strategy of having multiple buckets of physical pages doesn't really solve the problem
dormito has quit [Ping timeout: 250 seconds]
<geist>
but it just allows them to favor high memory in cases where they know they dont need to DMA, and thus make it more likely they'll have low memory when they need it
<geist>
but doesn't keep low memory from being exhausted
<mrvn>
it helps at boot so nothing grabs all the low pages first, which is where you most likely need special memory.
<klange>
the thing i love about doing things as a hobby
<klange>
is that my strategy for this sort of thing
<klange>
is *shrug*
* klange
walks away
dormito has joined #osdev
<junon>
mvrn: that answers like three questions all at once
<junon>
makes way more sense now, thanks everyone lol
<mrvn>
klange: call me again when some of my hardware needs it :)
<junon>
The article I'm reading way overcomplicates the DMA thing.
<mrvn>
junon: if you want to support the Raspberry Pi then you will have to care by the way.
<junon>
Is it the same idea though? It'll always have to be lower pages, right? In which case I can do page swapping as a last resort, right?
<mrvn>
Anyone know if the MALI gpu on basically every other ARM SOC does 64bit?
<mrvn>
junon: yes. It's all about having older hardware with a smaller address bus or without scatter gather and iommu capabilities.
<junon>
Right okay, makes sense.
<junon>
Thank you again :) very helpful
<geist>
and if you get Really Old, the legacy ISA DMA stuff only understands 1MB
<geist>
ie, floppy disks, old soundblasters, etc but you dont hear much about that around here because thats truly legacy
<mrvn>
At least here people still know what a floppy disk is. :)
<mrvn>
You know, that thing that plays Happy Birthday in a scratchy tune.
masoudd has joined #osdev
<geist>
dont need dma for that!
<klange>
wait does the floppotron have disks in its drives, I thought it was just the drive motors...
<klange>
paweł's last upload was six months ago ;-;
JerryXiao has quit [Quit: Bye]
JerryXiao has joined #osdev
Jari-- has quit [Remote host closed the connection]
<junon>
Looking at the buddy allocator, the examples I'm seeing are all very contrived and work well given the specific order of allocations. But I'm curious about allocations that are e.g. (1 o2 + 1o0) in size (where o2 = order 2, o0 = order 0 = 1 page). In such a case, I'm requesting 4+1=5 pages, won't I eat up a total of 8 pages (wasting 3)? In the case where my orders go up to cover the entire memory space, doesn't that mean (1 oN + 1o0) where oN is the highest order
<junon>
effectively waste half the memory space?
<junon>
Or does an allocation just need to be a multiple of o0 size and the allocator continues chipping away at the total allocated size or something until the entire allocation is satisfied? Seems incredibly expensive, but I guess it fits allocations the most optimally eh?
alpha2023 has quit [Read error: Connection reset by peer]
alpha2023 has joined #osdev
rb has quit [Ping timeout: 240 seconds]
<moon-child>
junon: yes, all allocations are rounded up to the next power of two
<junon>
That seems outstandingly wasteful though
<junon>
:D
<junon>
unless I'm missing something obvious
GeDaMo has joined #osdev
<moon-child>
say, 25% fragmentation, average case
<moon-child>
modern allocators promise ~10%
<moon-child>
so not _too_ bad, I think
rwb has joined #osdev
<junon>
Doesn't linux still use buddy though for page frame allocations?
<moon-child>
dunno
<junon>
that means typical case a linux server is mis-handling 25% of memory? xD
<moon-child>
those tend to be rather large, though. And userspace can know to allocate pot sizes
<moon-child>
which would waste nothing
<junon>
That's assuming userspace is playing nicely though.
<zid>
userspace plays nicely
<zid>
it basically only ever mmaps huge chunks which it feeds to malloc, in practice
<moon-child>
(I will also note: many userspace dynamic container implementations, such as hash tables and arrays, grow themselves in powers of two or otherwise exponentially. So if it really is a problem, it's not an isolated one)
<klange>
> page frame allocations
<klange>
userspace doesn't care, generally, unless it wants direct access to memory for DMA operations, as discussed previously
<junon>
AHHHHH
<junon>
right
<junon>
userspace allocations can be fragmented
<junon>
Okay so it's really only if DMA allocation or the kernel itself allocates a "rude" amount of memory that wastes a lot of the buddy block
<junon>
otherwise userspace allocations will fall cleanly into the o0 blocks?
<junon>
(o0 = order 0)
<junon>
thanks klange, I knew I was missing something obvious lol
<klange>
we are, once again, back to me saying I really do just use a dumb-as-bricks bitmap allocator for page allocations and it works _great_ for me :)
<klange>
my virtual address allocation could use some work... it's just sbrk behind the scenes, and now my malloc is unmapping pages when large allocations get freed, so if you do a lot of malloc+free of large allocations you can run off with the pointer...
<klange>
you could do that before, but you'd run out of RAM first
pretty_dumm_guy has joined #osdev
ElectronApps has joined #osdev
Payam60 has joined #osdev
dormito has quit [Quit: WeeChat 3.3]
<ddevault>
woot woot I am in long mode
zaquest has quit [Remote host closed the connection]
<j`ey>
gg
<ddevault>
on real hardware too
dormito has joined #osdev
zaquest has joined #osdev
<klange>
neat
<zid>
neat indeed
<GeDaMo>
The journey of a thousand miles begins with a single switch to long mode :P
<ddevault>
I can assure you that it was a journey of a thousand miles just to get to long mode
<zid>
did you go a particularly long way?
<zid>
I went.. a mildly long way in that I made a pmode ELF loader first
<ddevault>
no, I just ran into all kinds of frustrating issues
<ddevault>
aha, I'm about to write a long mode ELF loader
<zid>
frsturating issues is how I ended up with a pmode elf loader
<klange>
my x86 BIOS loader runs mostly in protected mode, and my x86-64 kernel has the longmode bootstrap from there as well as a specialized version for APs
<zid>
It ended up with the simplest makefile setup and could be understood by the most tools and booted by the most things
<ddevault>
I'm actually basing some of my work on your bootloader, klange
<ddevault>
at least your stage 2
<klange>
the "stage 1" was an afterthought, as I had originally written it purely for El Torito
gog has joined #osdev
<zid>
I simplified the makefiles for my bootstrap crud recently which felt nice
<zid>
it's now just gcc boot/*.c -o boot; gcc src/*.c -o kernel; mkisofs boot kernel fs -b grub/stage2
<zid>
effectively
nyah has joined #osdev
Burgundy has joined #osdev
Jari-- has joined #osdev
srjek has joined #osdev
dennis95 has joined #osdev
ptrc has quit [Remote host closed the connection]
not_not has joined #osdev
<not_not>
Hai
not_not has quit [Ping timeout: 256 seconds]
xenos1984 has quit [Remote host closed the connection]
ptrc has joined #osdev
xenos1984 has joined #osdev
not_not has joined #osdev
<not_not>
Found 540 euro in bitcoin
ptrc has quit [Remote host closed the connection]
masoudd has quit [Ping timeout: 272 seconds]
<mrvn>
junon: YOu have to completly rething your idea of memory. You have 3 separate things: 1) physical pages (limited resource) , 2) virtual address space (basically infinite), 3) An MMU that can map physical pages to virtual addresses.
<mrvn>
junon: If you keep those 3 things separate in your head you can much better reason about memory stuff.
<mrvn>
junon: Note that halfway moden hardware can scatter gather. So you do not need 5 contigous pages of virtual memory for a 20kb DMA, you can tell it you have 8k here, 8k there and 4k over on the oder side and it will DMA the chunks.
<clever>
the 3d core on the pi4 technically cant scatter-gather, but it does have its own mmu, so you could use that to map the scattered pages the same way, so the 3d core thinks its contig
<mrvn>
junon: modern devices even have iommu which is one step better.
<clever>
yeah, an iommu can also be abused to do that to any piece of hw
<mrvn>
clever: The compositor hardware goes through the MMU?
<clever>
mrvn: nope, the 2d subsystem for drawing to the actual screen, doesnt go thru any MMU, so it must be contig in the physical space
<clever>
but a lot of software stacks use opengl for the actual composition, ignoring the proper composition hw, lol
<mrvn>
too bad. That would have been really nice if you could scatter gather your framebuffers.
<clever>
so you need your final composited frame to be in the physical space, but the source textures (each window) can be fragmented
<clever>
also, the framebuffers must be in the lower 1gig of the physical addr space
<mrvn>
oh, by 2D subsystem you mean the hdmi output?
<clever>
yeah
<mrvn>
(for example)
<clever>
the 2d and 3d layers are entirely seperate components
<clever>
the 2d system can do its own composition
<mrvn>
That's OK I guess. You only need 2 or 3 final framebuffers.
<clever>
yeah, you could staticly allocate those, and then just rotate thru them
blockhead has quit []
<clever>
but the 2d system can acept multiple framebuffers for a single frame, and composite them together for you
<bslsk05>
'HVS scaling animation test' by michael bishop (00:00:26)
<mrvn>
seen it
<clever>
with support for per-pixel alpha
<clever>
all of that needs the images to be contig in the lower 1gig of ram
<clever>
but the 3d core, is a pure memory->memory device, that just turns shaders+textures+polygons into a final 2d image
epony has quit [Quit: QUIT]
<mrvn>
The RPi might be the worst case of memory I want to support. I have graphics memory (<1GB), DMA32 (<4GB), DMA (contigous, do I actually need this) and general use.
gildasio1 has joined #osdev
<clever>
the 32bit dma core, is also limited to the lower 1gig
<clever>
that 4gig address space, is just 4 aliases of the same 1gig
<clever>
but the dma controller takes a linked list of control messages, so you can do scatter-gather with it
<clever>
just create 1 control message per fragment
<mrvn>
Yeah, but I want to encode the intention that something is graphics memory because GPUs often have their own memory.
<clever>
linux uses a CMA pool for that, a large block of dedicated memory for contig physical allocations
gildasio has quit [Quit: WeeChat 3.4]
gildasio1 is now known as gildasio
<mrvn>
One idea I have floating in the back of my mind is that devices provide an abstract MemoryHint
<mrvn>
The MemoryHint must then be passed to alloc to get memory suitable for the device.
<clever>
the pi4 3d core is a bit of a complication, because inputs (textures) can be fragmented, if your willing to re-configure the mmu on context switch, but outputs usually cant be fragmented, but it depends on what will be consuming the output
<clever>
so its more then just what will be writing, but what will be reading it later on
<mrvn>
The idea would be that high level drives or apps don't need to care about what DMA engine is used, the DMA engine driver tells it.
<clever>
using dma for scatter-gather gets a bit more complex, if both sides are scattered
<clever>
because its basically just a hw accelerated memcpy, you need to find the longest block, for a given offset into both streams
xenos1984 has quit [Remote host closed the connection]
xenos1984 has joined #osdev
kkd has quit [Ping timeout: 272 seconds]
kkd has joined #osdev
<mrvn>
you mean split everything up whenever source or dest skips to the next location?
<mrvn>
Since you don't have separate "scatter lists" for source and dest but one list of source+dest pairs?
<clever>
exactly
<clever>
each entry in the control list is just a source, dest, and count
<mrvn>
easy for me. I just *shrug*, everything is 4k.
<mrvn>
Having any contig memory is probably so unlikely that it's not even worth checking.
<clever>
but for peripherals like the SD interface, the special no-increment flag is set on the peripheral side, so it hits the FIFO register every time
<clever>
which will either be src or dst, depending on the direction of the copy
<clever>
the dreq mode is also enabled, so dma throttles itself to the right rate
<mrvn>
Your saying that SD card will be faster with contigous pages?
<clever>
the dram can do a theoritical max of 25 gigbit/sec
<mrvn>
And the SD card 10-80MB/s.
<clever>
but the SD card can only do 200mbit/sec
<clever>
at the interface/protocol level
<clever>
less if the flash cant keep up
<clever>
(all numbers are from a pi0-pi3 model)
<clever>
so the slight latency spikes from dma having to switch pages, are nothing
<clever>
the only benefit, is that you know one side is never fragmented, because the addr never even increments
<clever>
SD reads are always from the fifo to the scatter list, so one control-msg per fragment
<mrvn>
I'm not getting what you are saying? ... Ok, the opposite of what I feared. I kind of assumed the memory was so fast the SD controller wouldn't even notice the scatter gather and then you started talking about that as a problem.
<clever>
SD writes are always from the gather list to the FIFO
<clever>
so your never doing both scatter and gather at once
<clever>
most peripherals (spi, sd, i2c, uart) are just a dumb FIFO, and a dreq line telling the dma if the fifo is above/below some hard-wired level
<clever>
relatively few (usb, 2d, 3d) are bus masters, that can do their own dma, and have different rules
<clever>
one major difference between them, is predictable io requests, and speed
<clever>
usb lets you queue up a different addr+size for each endpoint, so you dont know which request its going to do next
<mrvn>
NIC hopefully too
<clever>
pi0-pi3 has no on-soc NIC
<clever>
its just a usb device
<mrvn>
on non-RPis
<clever>
pi4 genet i believe is dma capable on its own
dude12312414 has joined #osdev
<clever>
yeah, in the x86 world, a dedicated memcpy hw block is more rare
<clever>
and every device just does its own dma
<mrvn>
If the NIC isn't DMA capable it would need it's own memory. With DMA it can just stream incoming packets into the DMA engine.
<clever>
not sure about other arm devices
<attaboy>
arm of king leoric
<mrvn>
Or your IRQ driver would have to be real quick in starting the DMA before the frame is lost.
vin has quit [Quit: WeeChat 2.8]
vin has joined #osdev
gildasio has quit [Remote host closed the connection]
dude12312414 has quit [Client Quit]
gildasio has joined #osdev
<attaboy>
GeDaMo: so i don't need to feel bad for not knowing what im doing if im startign to program for about a month ago right ?
<GeDaMo>
I /still/ don't know what I'm doing :P
<attaboy>
i refuse to accept that
<attaboy>
you are my all time hero
<CoffeeMuffin>
yes, in fact, you can be proud that you do not fall for that cognitive bias known as the Dunning-Kruger effect.
<bslsk05>
playlist 'Making an 8 Bit pipelined CPU' by James Sharman
<not_not>
Oooh
hbag has joined #osdev
<not_not>
That was beutifull
<not_not>
Cant wait til dark falls bloody eplepsia
<not_not>
Im like a vampire got photosensitive eplepsia
<not_not>
And i know living in the digital world is bad for my eyes but god damned it
<heat>
i've thought about designing a PCB around the rpi2040
<not_not>
Ooh
<heat>
it's pretty well documented
<not_not>
I miss my rpi
<j`ey>
heat: pcb design is fun
<not_not>
But good news i might have much funding around the corner
<Bitweasil>
The rpi2040 seems like a properly solid little board.
<Bitweasil>
I've got a stack of them to play with, just haven't gotten the time yet.
nyah has quit [Quit: leaving]
nyah has joined #osdev
<heat>
j`ey, yeah it's just a matter of actually knowing how to do it
<heat>
i've learned a bit about the process but i've never actually done it
<not_not>
First summer job i soldered pcb's
<j`ey>
heat: the hard part would be soldering the rp2040
<not_not>
That was a good summer
<not_not>
Imma try and detach myself from the need to sleep
<not_not>
48 hours of awake
<not_not>
Just coding
<heat>
and other hilarious jokes you can tell yourself
nyah has quit [Client Quit]
nyah has joined #osdev
<not_not>
Not is not a never ending recursive not joke
<not_not>
Not! It is not not a never ending recursive not joke either
X-Scale` has joined #osdev
mahmutov has joined #osdev
X-Scale has quit [Ping timeout: 272 seconds]
X-Scale` is now known as X-Scale
<Bitweasil>
j`ey, why? It's got standard header pin spacing on it.
<j`ey>
well if youre not used to soldering something that small, that is
<Bitweasil>
Er, are we talking about the *chip* or the full Pi Pico?
dormito has joined #osdev
<Bitweasil>
The Pi Pico has the 0.1" headers.
<Bitweasil>
My daughter's 7 and can solder those.
<not_not>
Hahaha she living the dream
<Bitweasil>
If you're talking about the actual SoC, yeah, that's pretty fine pitch.
<j`ey>
I assumed the chip
<Bitweasil>
Oh, I assumed this was a "Put a Pi Pico as a daughterboard on a PCB" project.
<not_not>
Did u just asume my chip?
<j`ey>
Bitweasil: that would be easier!
attaboy has joined #osdev
<Bitweasil>
Sorry, that's just what I assumed the solution here was - the Pi Pico is a perfect mezzanine board for a larger PCB.
<Bitweasil>
They're flat on the underside and have the castle pins on the edge, you can just literally plop it down on an insulated chunk of PCB and solder it down.
dormito10 has quit [Ping timeout: 256 seconds]
<j`ey>
and not expensive
<Bitweasil>
Correct.
<Bitweasil>
If you're doing something extremely custom, yeah, the bare 2040 chip is worth using, and they're dirt cheap too, but for any sort of initial project, why not use the power-and-breakout board that's also quite cheap?
<heat>
yeah I was talking about the soc
<heat>
as a learning thing
<Bitweasil>
*nods* You could hand solder it.
<heat>
i also don't know how to solder at all, which is kind of a problem
<mrvn>
don't solder SOCs then.
pretty_dumm_guy has quit [Quit: WeeChat 3.4]
<Bitweasil>
Ah, yeah. Learning to solder a Pi Pico down would be doable, not the fine pitch SoC soldering as your first project.
<mrvn>
You either create shorts or overheat the chip.
<GeDaMo>
Don't they use solder paste and reflow for SMDs?
<mrvn>
totaly.
<not_not>
Heat u just apply heat to solder using a soldering iron
<Bitweasil>
GeDaMo, typically, yes, but you can also hand solder them.
<heat>
thanks
<mrvn>
You can buy PCBs and have them solder on the SOC and other chips for you and then you just finish the easy bits.
<heat>
i now know how to solder
<Bitweasil>
Most of them, and the pitch on the Rpi2040 isn't that bad.
<mrvn>
CoffeeMuffin: The access API most filesystems have is that each node has a number (inode) and you can access nodes by their inode number. So the whole tree/graph gets flatened into big hashtable or tree.
<mrvn>
Depends on your OS if that is the way you want to go too.
<mjg>
GeDaMo: ye
<geist>
oh yeah that's a weird corner of PC i haven't explored: el torito
<CoffeeMuffin>
that makes sense mrvn! So it is just a big associative array.
the_lanetly_052_ has quit [Ping timeout: 256 seconds]
<geist>
to respond to a conversation 7 hours ago
<geist>
fun fact, the guy that wrote that book went on to write APFS
<mrvn>
CoffeeMuffin: it's a simple starting point. You run into problems with fragmentation over time. A tree might be better because you can spread it out all over the memory wherever you find a free page.
<mrvn>
Or you can implement a defrag operation, e.g. keep the associative array at the start, the files at the end and when the two meat you defrag and moves the files back.
<mrvn>
Note: In this implementation directories are just files (or blobs your array points to) too. They contain a list/tree/hashtabl/whatever of filename and their inode numbers.
<CoffeeMuffin>
you do not even need directories with associative arrays
<CoffeeMuffin>
because the name foo/main.c is different from bar/main.c
<mrvn>
from a users perspective they want their filesystems to have directories.
<CoffeeMuffin>
yeah, you could do that on an abstraction level above
<mrvn>
Oh, don't do that. Consider what happens with "mv foo blub". Do you want to rename all the files?
<mrvn>
And you have ., .., symlinks, hardlinks. Logcially you want a tree. It's just the memory representation of that tree is an associative array.
<CoffeeMuffin>
mv foo blub would mean there is a file named foo in the root directory, and the same for blub
<mrvn>
not what the user expected. foo is a directory.
<mrvn>
If you "mv foo blub" the user expects to have a file blub/main.c
<CoffeeMuffin>
you are right
<CoffeeMuffin>
if it was just an associative array you would have to search for all files that are foo/* and rename them to blub/*
<CoffeeMuffin>
which is obviosuly not good
<mrvn>
No. you have to separate the logical representation as a tree and the memory representation as an array.
<mrvn>
Take your tree. Leaves are files with some data block attached. Then directories are Nodes with children, each a pointer to another Node.
<CoffeeMuffin>
exactly
<mrvn>
Now replace the pointer with an index into your array instead.
<mrvn>
So your filesytem has nodes 0-10000. Each can be a directory, a file, empty, (a socket, a device, a pipe, ....)
<mrvn>
That's the verry old and basic design of an inode table and data blobs.
<mrvn>
It's still a tree but now it's flattened into the array.
<mrvn>
Or just ignore the fact that you have persistant memory and just do a in-memory filesystem as a simple tree. Write an allocator for your persistant memory and use that to allocate the tree.
<mrvn>
Inode numbers can be the address of the Node and you are compatible to existing filesystem APIs.
pretty_dumm_guy has joined #osdev
pretty_d1 has quit [Ping timeout: 240 seconds]
<geist>
yah the allocation scheme for assigning inode # can vary, but it's a nice solid design to have directory entries point to an inode # instead of being the same thing as an inode (ie, FAT)
<geist>
even very non unixy fses like NTFS do fundamentally the same thing
<geist>
their inode # are just an index into the MFT (master file table)
<mrvn>
can't do hardlinks if a directory contains the actual inodes.
freakazoid343 has joined #osdev
<mrvn>
plus cache locality. A inode # or node address is way smaller than the whole inode making your lookup table more compact.
<mrvn>
Anyone here unit testing and cross compiling their kernel? Do you build everything twice or run unit tests in qemu?
<geist>
i build twice, with a -test version of the build
<geist>
but it only takes like 20 seconds to build so not too worried about it
<mrvn>
a native or cross build?
<geist>
cross build
<geist>
sice the kernel runs onlike 9 arches
<mrvn>
and then qemu-user-arch?
<geist>
yah
<CoffeeMuffin>
Thank you mrvn! Now I am properly falling down the file system rabbit hole.
<geist>
adding more unit tests as i go (relatively new thing) but there's a command to run the unit tests and can search for a pass/fail
<geist>
CoffeeMuffin: yay. FS rabbit hole is a good hole. was my first job
<geist>
still has a place in my heart
<mrvn>
Compile times for me are still measured in seconds overall. So I'm not concerned with running unit tests on every compile taking too long yet.
<mrvn>
Just have to add tests first.
<geist>
yah, 20 seconds is actually on a really slow machine. on a fast machine it's like 3
<geist>
yah. i was not a huge fan of unit tests before, but we use it like crazy on zircon and i'm grudgingly accepting that at least some set of tests are useful
<mrvn>
And later you only change a subsystem and don't have to rebuild everything. So compile times should remain small.
<geist>
i just dont believe in writing silly unit tests, but that's fairly subjective what is useful and what is silly
<mrvn>
ideally I want some code coverage and have it fail if some code isn't covered I think. Even if that needs some silly tests.
<mrvn>
Even if just to test it out and see that I don't like it.
<geist>
yah one of the biggest drags i've seen is having lots and lots of unit tests that get in the way of future refactors of things
<geist>
but so it goes.
<geist>
for a large organization thats not a concern, it's generally a non-goal to make it easy to refactor code
<geist>
for a hobby thing, you might want to keep yourself fast and free, so you dont get your momentum stopped by some silly test
<mrvn>
The unit test should test the public API. So if they get in the way of refactoring then that means you have tons of users of the unit that need to change too.
<mrvn>
I've seen people add tests for internal functions but I think that is wrong. You have to find valid inputs to the public api that tests all your code. Otherwise you end up with internal functions that work on things that never happen and refactoring becomes hard.
mahmutov has quit [Ping timeout: 250 seconds]
<geist>
right, that's part of the subjectiveness of what is useful and what is silly
<geist>
agreed in most cases with you
<mrvn>
My feeling is the tests are supposed to make sure that after refactoring the code still fullfills the contract amde to the users of that code.
<geist>
ie, having a test on some thing, say a heap, that allocates a bunch fo stuff and then internally walks over the data structure making sure it looks a particular way may appear to be very thourough
<geist>
but then it really gets in the way of future refactors and is very 'brittle'
<mrvn>
and if you change the test when you refactor then it doesn't say anything about compatibility. It just says the new code follows the new contract.
<geist>
but on the same token, having a unit test that say tells the api to the MMU code that says 'map these 20 pages ni this way' and then having a unit test that walks over the page tables to verify they were set up a particular way seems okay
<geist>
since the API basically mandates the page tables be in a particular way
<mrvn>
use functions. The unit test should call a walk_table function but not implement one.
<geist>
right
<Bitweasil>
I've generally done stuff like that as "Ensure the result is expected." For MMUs, put a bunch of values in physical memory, make sure that what I map virtually returns the expected result from physical memory.
<mrvn>
can get kind of silly with handwaving while doing the same thing. But it adds a bit of separation between the test and the implementation details.
<geist>
anyway initially when exposed to an agressive unit test mentality when i joined google i was pretty turned off by it
<Bitweasil>
For a heap test, filling each allocated region with a unique value, and ensuring that the stuff remaining doesn't get stomped after some free/alloc cycles would seem useful.
<geist>
but i think it was because i didn't have a nuanced view of it initially
<geist>
i lumped most unit tests into the silly/extraneous category
<Bitweasil>
I think Google's test driven mentality is right, having worked plenty of places without it.
<mrvn>
Bitweasil: or do a heap sort with a large random array.
<Bitweasil>
And I've kept that into my current position.
<geist>
and was fixed on the notion that by using unit tests everyone was ignoring all other forms of tests
<geist>
but that's not really tru
<mrvn>
My problem is that I have never done unit testing so how do I start now.
<geist>
Bitweasil: yeah agreed. as much as it's painful to admit, because i still to this day hate writing unit tests
<Bitweasil>
It makes it an awful lot easier to test early in development.
<Bitweasil>
mrvn, GTest tools and go from there.
<mrvn>
Bitweasil: for your own kernel?
<geist>
i've always been a huge fan of assert early, assert often and generally make code brittle to anythig being out of place
<Bitweasil>
No, for work. Currently writing a GICv1/v2.
<geist>
but even that i've worked in orgs (*cough* apple) where that was explicitly not what you were supposed to do
<Bitweasil>
I'm down in this space, or below, but don't actually have a hobby OS.
<geist>
which i thought was really strange
<Bitweasil>
Apple's QC lately has been chasing after Microsoft. "What, us, test? That's what users are for!"
<mrvn>
Nowadays I often go a step further and explicitly build in canaries.
<geist>
my guess is it was just the kerne team at the time, and it has changed, but the kernel team explicitly had a rule of 'dont assert if the kernel will jsut crash anyway'
<Bitweasil>
My main problem right now is that implementing stuff, it's entirely possible to "symmetrically misunderstand the manual."
<geist>
ie, no null pointer tests, no out of bounds tests, etc
<Bitweasil>
o.O
<Bitweasil>
Uh.
<geist>
because the kernel will just crash so you can just debug it when you do
<Bitweasil>
Thar be security holes on the horizon, cap'n!
<geist>
because asserts are slow, etc etc
<mrvn>
geist: If the kernel just reboots the user will just go on. If it shows an error message then they will complain.
<geist>
yah i thought it was dumb
<geist>
my experience is assert early and often so you can get good, actionable error messages and skip a lot of debugging
<geist>
and make module A brittle if module B starts misbehaving and coloring outside the lines
<Bitweasil>
I can see an argument for having a debug and release build, where asserts are ignored in the release build.
<geist>
i think it has worked quite well in zircon at least
<mrvn>
currently I'm trying for making constexpr/consteval tests. So code simply fails to compile.
<Bitweasil>
But "dmesg says this impossible thing happened, and the kernel is going to crash" is a lot easier than "The kernel crashed, go look at the backtrace."
<mrvn>
Bitweasil: I can't. asserts are dirt cheap.
mahmutov has joined #osdev
<geist>
yah that too. a fair amount of the zircon unit tests compile down to 'return true;' because they're basically statically resolved
<geist>
which is okay
<geist>
we have two levels of asserts in the kernel, came from LK: ASSERT and DEBUG_ASSERT
<Bitweasil>
I've growled at a few people at work about writing tests, because they've written some code that (say) gcc does fine, clang fails on.
<Bitweasil>
Vector stuff.
<geist>
the former is on always, the latter is on for debug builds
<mrvn>
geist: I want them to compile down to static_assert. Only when that can't be done delegate to a unit testing framework.
<Bitweasil>
"You'd better make sure this fails to build on a compiler you know is bad..."
<geist>
though the difference in performance is only a few percentage on some benchmarks
<geist>
that may not have been the case 10-20 years ago, to be fair, but modern machines eat up compare and branches for breakfast
<geist>
especially hinted unlikely branches
<mrvn>
especially with [[likely]]
<mrvn>
+un
<Bitweasil>
Yeah, I tend to use that on sanity checks in code.
<Bitweasil>
if (unlikely(value >= array_size))
<mrvn>
With [[unlikely]] gcc will move the test code out of the function too. At least I've seen it do that with exceptions.
<geist>
a trick i picked up years ago in BeOS that has still worked to this day is to put magic values at the top of most critical structures
<geist>
and assert they're still there
<Bitweasil>
I want it to not corrupt a data structure, but neither can I *guarantee* that users won't be stupid with hwo it's used.
<geist>
also useful for use-after-free because the destructor zeros it out
<Bitweasil>
Oh, I *like* that!
<Bitweasil>
I may have to start doing that.
<mrvn>
Bitweasil: use std::span
<Bitweasil>
lol, C++.
<Bitweasil>
I write in C.
<geist>
yah in C you have to do it manualy, but it also is useful when you are using a hex editor to walk through data structures
<geist>
i use it in LK all over the place so when we converted to C++ in zircon we built a little class out of it that you try to put at the top of your class
<geist>
it doesn't catch corruptions in the middle of your object, etc, but it catches a fair amount of things
<mrvn>
I also like having void free(void **ptr) { ::free(*ptr); ptr = nullptr; } kind of free.
<geist>
so the instant we start seeing a corrupt thread class, for example, you start to suspect someone is stomping on things
<Bitweasil>
*nod* I tend to do that explicitly if I'm freeing something.
<mjg>
i dislike the nullification
<mjg>
set it to a poison
<mrvn>
mjg: same thing.
<mjg>
no
<mrvn>
something that will fail on use-after-free
<Bitweasil>
like NULL?
<mjg>
various code legitimately does if (foo->bar != NULL) ... which will appear to "work" if you just set it to NULL
<geist>
an explicit poison would tell you it was a use-after-free instead of something unnitiziaed
<geist>
ie 0x99 or something
<Bitweasil>
Hm, actually, a non-canonical address would work well in x64.
<mrvn>
mjg: but if the code does that then they are prepared to work with moved-out-of objects. That's correct.
<mjg>
best thing to do is to get kmsan or so and mark the pointer as crap
<mjg>
mrvn: no, you are hiding a use-after-free from that code
<mjg>
for example consider calling the destructor twice
<mrvn>
mjg: have you used move semantic a lot yet?
<geist>
and yeah we have KASAN now and it has caught a few thins, but setting that up was not easy
<geist>
i dont think it's something hobbyist osdevers would probably have the time/energy to do unless that really floats your boat
<mjg>
mrvn: no, but i ran into my fair share of bugs hidden by foo = NULL on free
<mjg>
geist: afair kmsan et al ports are not *that* difficult, but i only skimmed through what freebsd did
<geist>
that does remind me, i should change our 'canary' code (the magic stuff) to write instead of a 0 to it, the inverse of the magic
<geist>
that way could tell the diff between corrupted and use-after-free
<geist>
mjg: yah there was some really hairy bootstrapping issues though
<mrvn>
mjg: with move semantic you have a lot of "other.data = data; data = nullptr;" Kind o9f goes into the same boat. A dedicated value for not-initialized, moved-out, freeed, nulled, ... makes sense.
<geist>
and some clever use of page tables and whatnot, since you have this gigantic sparse array
<mrvn>
Maybe some trickery with using the top 8 bit of a pointer on ARMv8?
<mrvn>
So it's all NULL but different kinds of NULL.
<geist>
indeed. that's a fun thing. we're actively turning that on for zircon right now, though only for EL0 up front
* Bitweasil
makes unhappy noises about all that pointer stuff...
<mrvn>
what abvout the address authentication?
<geist>
are you asking me?
<mrvn>
Not sure exactly what it does but sounds like the top 8 bit of addresses become some kind of hash/token to validate the pointer was constructed right.
<mrvn>
geist: yes
air has quit [Ping timeout: 240 seconds]
<geist>
right, at the hardware level it simply tells the hardware to ignore the top 8 bits. bit 55 becomes the 'sign bit
<geist>
ie, determining what aspace to match against (TTBR0, TTBR1)
<geist>
in user space the compiler will generate code to test 8 bit tags against whatever
<Bitweasil>
Mostly, it's just going to be a pain to write, if I need pointer authentication and stuff.
<mrvn>
geist: no, that's the base thing to just ignore. There is another mode on top to have those bits have meaning.
<geist>
it's a compile thing
<geist>
mrvn: that's MTE, that's a different thing, and i thik incompatible
<Bitweasil>
There are about 4 different things you can do with "upper address bits" in ARMv8.
<Bitweasil>
SOme compatible with other, some not.
<Bitweasil>
Yeah.
<geist>
or whatnot. that's also not generally not implemented in most hardare
<Bitweasil>
And you can configure the size of that region too.
<geist>
wheras TBI (top byte ignore) is mandatory in armv8.0
<Bitweasil>
So it can be larger than 8 bits, in some cases.
<mrvn>
Have to check if tha address authenticaction is required.
<Bitweasil>
I'm sure I'll grumble about it more in a month or two, going from userspace emulation to full system emulation requires all that stuff.
<geist>
again, this is *just* telling hardware to ignore. it's up to software to implement a pointer tagging scheme
<mrvn>
Don't think it was one of the optional extensions like MTE but that doc is so confusing.
<geist>
there's some scheme that llvm can generate tagging stuff that we want to turn on
<geist>
presumably it boils down to a bunch of inline tagging checks prior to dereferencing pointers. hoefully it's not too slow
not_not has joined #osdev
<mrvn>
if it's a compiler flag then you can always turn if off for release.
<not_not>
Hi
<geist>
mrvn: right
<mrvn>
defintely something I want.
<geist>
the ramifications to the kernel are that you get all these pointers in from user space that may have upper tags set
<not_not>
Is there any podcasts about osdev or programming language design?
<geist>
so you need to sanitize pointers prior to using them (or not)
<Bitweasil>
No idea, not a podcaster, sorry.
<mrvn>
have to test bit 55 instead of bit 63.
<Bitweasil>
I'd get some of the classic OSDev books, though.
<geist>
hmm, not sure i can think of a worse possible way to get osdev or programming language design information than to listen to someone blab about it
<Bitweasil>
Tannenbaum's still remains relevant.
<Bitweasil>
Modern Operating Systems, I think?
<not_not>
Bittweasil ty
<geist>
mrvn: yah also things like does hardware strip the tags on exceptions or not. and if you deliver an exception report to user space do you want to strip it or not, etc
<mrvn>
isn't there a "design" in there too?
<geist>
not a biggie, but a few places
<not_not>
Geist im trying to save my visual lobes
<mrvn>
I sure hope the execption does not strip the tag. I want usefull info in the tag.
<not_not>
Mrvn idk compilers are fun to make
<Bitweasil>
Huh, here's the 4th edition as a PDF...
<geist>
mrvn: i dont think they do, but it's of course spread across 10 different plces in the ARM ARM
<geist>
the FAR_EL1 usually holds the tagged pointer
<geist>
but if you then go and drill through your VM you probably want to strip it first, etc
<not_not>
Ooh nice its got operating sysyem history and appreciation
<mrvn>
geist: do you have any code coverage tool for zircon
<geist>
for the kernel no, for user space yes
<geist>
i think that's an active project, but something makes it hard to do
<not_not>
So, i have a hacker on my dektop at home and he evades wireshark even
<geist>
presumably the existing tools require some user space libs or whatnot that is hard to use in kernel space
<mrvn>
I wonder if gcc/clang have any options to output code coverage data for all the constexpr code.
<not_not>
One reason im into osdev is to debug his rootkit and steal it
<mrvn>
Otherwise any coverage of my code would get lots and lots of holes.
<geist>
not_not: if that's really true, we really dont want to have anything to do with it
<not_not>
Oh
<not_not>
Sorry
<geist>
we're not here to haxxor stuff nor do we actually promote any of that stuff
<geist>
though what do you mea you have a hacker on your desktop?
<geist>
(i wonder if i'll regret asking)
<not_not>
I just felt it when i came in the room
<not_not>
So i opened paint and drew a cartoon message
<not_not>
Came back from dinner and he had answered in paint
<Bitweasil>
I think you want #security.
<not_not>
Only way he can have come in is thru wotlk 3.3.5a game client
<mrvn>
geist: I already do
<Bitweasil>
And your time spent debugging the rootkit would be better spent with a nuke-and-pave.
<Bitweasil>
WOTLK?
<not_not>
Ahh ty i wont besmuge this chan
<Bitweasil>
Also, you might consider Qubes. ;)
<not_not>
Wrath of the lich king
<geist>
oh wow. yeah that or you're imagining things
<Bitweasil>
Wasn't that Warcraft III?
<geist>
unplug your computer if that's the case and retry the test
<not_not>
Dw he hasnt stolen anything
<Bitweasil>
... how would you know if he has?
<mrvn>
Hacker on your desktop? Get the big ax and cut the hardlink.
<not_not>
Missing money
<Bitweasil>
If I ran into a response like that, I would *immediately* unplug my entire infrastructure and rebuild from known-good images.
<Bitweasil>
(which would probably involve a trip to a local computer store to purchase some random laptop they had on the shelf I could use as a root of trust)
<Bitweasil>
(for downloading and validating OS images to start with)
<mrvn>
Bitweasil: don't you have some old DVD or USB stick with write protect on the shelf?
<not_not>
Ahh, i just let hackers in and find their control center and use it as leverage on them
<Bitweasil>
I probably have some OS images laying around, but I've also done security too long to trust firmware on a compromised system without reflashing BISOes.
<geist>
...
<mrvn>
On that note: Anyone know if there are M2.key or sata SSDs with a write protect switch?
<not_not>
Bitweasel yehhhhhhhh
<Bitweasil>
Not a bad idea, though - keep some DVDs with validated OS images.
<geist>
that's a good point
<not_not>
On that note im starting companied now so i cant play games with hackers anymore
<geist>
burn the disks ahead of time on a clean machine yeah
<j`ey>
I like the idea of driving to a PC World (Best Buy, for you americans :P) and buying a new computer for root of trust
<mrvn>
Bitweasil: a grml image is a good base. You might want to add some intrusion detection tools or ssh keys to it.
<Bitweasil>
mrvn, a lot of SD cards have write protect switches. Or, at least, the adapters do.
<not_not>
Im doing lfs for that very reason
<Bitweasil>
So put a uSD in an adapter and keep it in there.
<Bitweasil>
grml?
<not_not>
On an sdcard lol
<mrvn>
Bitweasil: I have an USB stick with clean image for that.
<Bitweasil>
Yeah, but I don't trust USB sticks. :p
<geist>
i suppose this is where something like an actual floppy disk is useful
<bslsk05>
'Microsoft Code Review Secrets' by Dave's Garage (00:18:34)
<geist>
what is this link?
<Bitweasil>
Dunno, it'd crash my browser if I clicked on it.
<mrvn>
Bitweasil: by that I mean database of known good hashes of the install and such.
<geist>
if it has something to do with looking through leaked code, please do not post that stuff here
<Bitweasil>
mrvn, it's quicker to just pave the systems with a fresh install. I don't do a lot of customization for this exact reason.
<geist>
let it be known that i have never nor will ever look at leaked stuff like that (for the record)
<not_not>
God damned it they messed with my hobby
<Bitweasil>
not_not, unplug network cable...
<not_not>
Ye
<geist>
defenestrate computer
<Bitweasil>
I could *probably* bootstrap trust with one of my less frequently used old laptops or something that's almost always shut down and disconnected...
<Bitweasil>
But I'm going to burn a couple OS ISOs now, because that sounds like a useful thing to have.
<Bitweasil>
I wonder if the ODroid N2+ can burn a CD... probably.
<geist>
haha that old dumb Dell mini 9 that i still have floating around
<Bitweasil>
*fiddles*
<geist>
The Most Useless Computer
<Bitweasil>
Yeah an old netbook or something.
<geist>
though it has no cdrom
<geist>
the bigger problem i guess is none of my modern cputers have cdrom drives
<Bitweasil>
I have a few USB CD/DVD/BD drives.
<geist>
should probably get a USB cdrom drive before they're unobtanium
<Bitweasil>
They're used so infrequently that I'd generally trust them.
<Bitweasil>
Get a BD burner one, they're not very expensive, and 100GB on optical is enough to be properly useful.
<geist>
yah i have a BD burner, should probably just find an external USB case
<geist>
especially sine i never use the burner anymore, it's not even in a computer i think
<not_not>
Im doing linux from scratch build now
<geist>
if your'e doing it plugged into the network then you're already screwing up
<geist>
air gap it
<not_not>
How u gonna protect from entanglement attacks tho?
<freakazoid343>
geist, If you were referring to my link its about best practices.
<geist>
okay good. false alarm
<geist>
it just had a dubious title so i didn't know what it was about
<not_not>
Hmm im gonna paint in the dark
<not_not>
Cya latee
<Bitweasil>
later!
* Bitweasil
grabs a couple OS ISOs to burn.
<Bitweasil>
I do wish there were a way to download a Windows ISO on Windows without that stupid "Media Creation Tool."
<Bitweasil>
It's so nice coming from Mac or Linux. "Ok, here's the ISO you wanted, download it."
<Bitweasil>
Vs "Download this tool that can be coerced into making an ISO."
<geist>
yah dunno if there's simply some url behind it, but if it is it seems to move around
<Bitweasil>
It seems to have some auth token.
<Bitweasil>
For what reason, I have *no* idea.
<Bitweasil>
I guess so you end up always having to go through the menus and get a current ISO?
<Bitweasil>
It doesn't make a lot of sense, really.
<bslsk05>
www.microsoft.com: Download Windows 10 Disc Image (ISO File)
<GeDaMo>
Win10_21H2_EnglishInternational_x64.iso
<Bitweasil>
Are you on Windows or not-Windows?
<GeDaMo>
Linux
<Bitweasil>
Yeah. Try that page on a Windows machine.
<GeDaMo>
Yeah, it did say something about a redirect
<Bitweasil>
It'll only let you download the Media Creator Tool.
<Bitweasil>
Which can eventually emit an ISO, but... still.
<Bitweasil>
"plz2have weblink?"
<GeDaMo>
Which language do you want? :P
<Bitweasil>
I'm good.
<zid>
czech
<mrvn>
Bitweasil: growisofs https://...
<mrvn>
Bitweasil: Will you like the new RPi4 firmware that lets you just download and install from the firmware?
<Bitweasil>
I think it's a slick trick, yes.
<Bitweasil>
I'm honestly less interested in the Pis lately, though.
<Bitweasil>
I still have a bunch, but they're bloody well unobtanium, and the other ARM SBCs are a bit more exciting.
<Bitweasil>
I've got a N2+ or four, and other than RAM, they're a good bit nicer than the Pi.
<Bitweasil>
My PineBook Pro is excellent.
<Bitweasil>
And I've got a set of Rock5 boards on order - quad A76 + quad A55, NVMe, 16GB RAM.
<Bitweasil>
Which, if they're *remotely* close to what I hope for, should be everything I've wanted in an ARM SBC for about 4 years.
<Bitweasil>
That's not a "light desktop" anymore, that's just a desktop.
<mrvn>
yeah, bt do I want to spend 200EUR now for a system I might use in 4 years? :)
<Bitweasil>
Eh?
<Bitweasil>
I'm saying, for the past 4 years, I've wanted ARM SBCs with >4GB of RAM.
<geist>
yah the only real nice one that'd be interested as a desktop is the rpi 400
<Bitweasil>
The 8GB Pi4 is nice, but the CPU is a bit slow and the lack of eMMC or such for storage is annoying. USB3->SSD isn't awful, I've been using that for years, but there are now some interesting other options.
<geist>
and its neat but mostly that
<Bitweasil>
And that's only 4GB.
<mrvn>
Bitweasil: perfect for developing the kernel and testing 64bit + >4GB ram. it's cheap.
freakazoid343 has quit [Read error: Connection reset by peer]
<Bitweasil>
The Pis are great, I love the support on them, but as I'm using them for more desktop use, I do hit limits.
<geist>
also rpi4 8GB is a great KVM host
<Bitweasil>
"cheap"?
<Bitweasil>
Go find me an 8GB Pi4.
<mrvn>
Bitweasil: the Rock + 16GB ram is too expensive for me as a test system.
<Bitweasil>
They're $150-$200, if you can find one.
<geist>
sure, but that's because they're unobtanium
<mrvn>
Bitweasil: got my last when they still had them cheap.
<zid>
for $200 you can almost build my desktop
<Bitweasil>
I recognize my irrational hatred of Intel and x86 isn't the most sane thing...
<mrvn>
zid: other than the PSU and case the rock board is your desktop.
<Bitweasil>
But I've been enjoying as the ARM SBCs go from "toy you can make work" to "legitimately powerful."
<Bitweasil>
I don't want a $5000 ARM tower workstation.
<mrvn>
Bitweasil: totally. But I already have a desktop and laptop.
<Bitweasil>
But $200 for 4+4, NVMe, and 16GB of RAM is entirely sane.
<Bitweasil>
IMO
* Bitweasil
has... uh...
<mrvn>
Bitweasil: the NVMe is extra though
<Bitweasil>
Honestly, I'd have to go count.
<Bitweasil>
mrvn, the port isn't.
<mrvn>
+ wifi module
<Bitweasil>
And a used 128GB NVMe isn't that expensive.
<mrvn>
and PSU, getting up to $300 probably.
<Bitweasil>
I'm unclear as to if it ships with the wifi module or not, but I am *exceedingly* happy it's removable.
freakazoid333 has joined #osdev
<mrvn>
The picture didn't have the module in it so I assume it comes without.
<Bitweasil>
"We'd like a Pi for ARM dev" runs often enough into "You can't, it has wifi. That's not permitted in this building."
<mrvn>
then get one without
<Bitweasil>
If you want to desktop a Pi you need a USB -> SSD adapter and SSD anyway, so kind of a wash there.
<Bitweasil>
The Pi3/Pi4 don't have a non-wifi option, do they?
<mrvn>
pi400? CM4?
<mrvn>
You just acre about 64bit core, right?
<Bitweasil>
The 400 doesn't have wifi?
<Bitweasil>
Mostly, yes. It's not been an issue for me personally, but I know a number of people who've been annoyed by that.
<mrvn>
I only see a wifi PI400 here.
not_not has quit [Ping timeout: 256 seconds]
<Bitweasil>
Anyway, quad A76s @ 2.4 should rather crush the Pi4 in compute as well.
<Bitweasil>
A...72s, 73s, at 1.5 (you can push them, but my Pi4 was getting glitchy pushed).
GeDaMo has quit [Remote host closed the connection]
<mrvn>
Bitweasil: you have to find a Raspberry Pi 3B, not 3B+.
<mrvn>
Used to be no problem to get a Pi without WiFi but now unobtainium
<mrvn>
Do you have to leave your phone and laptop at the entry when you enter the building?
xenos1984 has quit [Remote host closed the connection]
<Bitweasil>
I no longer work in such spaces, but, yes, I've done that for many years of my life.
xenos1984 has joined #osdev
<mrvn>
Bitweasil: did you oder from allnet or ameridroid?
<mrvn>
(and why?)
<Bitweasil>
Ameridroid.
<Bitweasil>
... and because that's where I found them.
<Bitweasil>
Prefer to support something resembling a more local shop.
<mrvn>
but but but, that will have CIA spyware on it instead of chinese spyware. :)
<Bitweasil>
*shrug* Probably.
<mrvn>
Kind of hate to pay US import and then EU import tax.
<Bitweasil>
If you're not in the US, probably don't order from Ameridroid? :p
<Bitweasil>
I live in the US.
<mrvn>
I figured. Ameridroid is a bunch more expensive.
<mrvn>
But probably the saame price if you add the US import tax to ordering from china.
<mrvn>
Do you know what the docs for the CPU look like? Are they understandable if you aren't geist?
<geist>
whcih one?
* geist
reads backwards
<mrvn>
Rock 5B RK3588
<geist>
oh no idea. but my past history with rockchip docs is they're extreely minimal
<geist>
like 'there are 4 cores, you know what those are'
<geist>
and 'here is al ist of irqs'
<geist>
basically a bunch of stuff you can figure out from a device tree
<geist>
but probably still easier to work with thatn broadcomm cpus because they're pretty standard. ie, they plug together off the shelf components, mostly from ARM
<mrvn>
That's better than the Alwinner A10 I looked at: The Alwinner has an 15560 like UART. *CONFIDENTIAL* With these registers *CONFIDENTIAL* location undisclosed.
<geist>
yah well, what i mentioned is just based on memory. you'll have to get the docs and look yoruself
<geist>
i am generally not interested in rockchip or allwinner stuff, but mostly because i just dont want to futz with Yet Another Basica ARM SOC
<mrvn>
Bitweasil: have you looked at it already or is this just for your desktop and you won't play with the hardware directly?
<mrvn>
geist: bitweasel mentioned the board. The 4GB model is $79, so quite affordable and I don't have a board with NVMe yet.
<mrvn>
+ big/LITTLE
<Bitweasil>
mrvn, I plan to use it as a desktop class system, so it'll be more "Does the kernel support what I care about?"
<Bitweasil>
I might hack on a second one, but I honestly don't do osdev as a hobby, just play in associated spaces for work.
<bauen1>
those rockchips that i looked at actually had a usable technical user manual, the allwinner manuals in some cases just don't mention some components and you're basically looking at multiple manuals for differnt chips to find all the puzzle pieces ...
<geist>
bauen1: ah good to know
<mrvn>
you make me hope.
<jimbzy>
That's what I was wondering, bauen1.
<mrvn>
If at least some chips havce good docs and it's off the shelf components anyway then everything I will use is probably easy.
<jimbzy>
I was going to ask, but I was also too lazy to scroll through the buffer to see if it had already been mentioned. :p
<bauen1>
mrvn: my standards for "usable" where quite low after looking at allwinner documentation
<mrvn>
bauen1: The A10 was the first chip I tried to hack on and boy did I have to works through linux kernels just to get hello-world printed.
<bauen1>
mrvn: that's one of the first allwinner chips isn't it ?
<mrvn>
I just wanted to know the physical address of the UART to bang some chars on it.
<mrvn>
bauen1: probably, it was years ago.
<bauen1>
mrvn: i managed to get UART 115200 baud printing working just by looking at the allwinner manual, so it has improved a _tiny_ bit
<Bitweasil>
mrvn, my goal, mostly, is to de-x86 my life as much as I feasibly can. As ARM stuff improves, I'll probably migrate my servers over eventually.
<Bitweasil>
Right now, my homeserver and datacenter server are both x86. :/
<mrvn>
Still have to work on amd64 for work.
<jimbzy>
Can't go wrong with the 486, Bitweasil ;)
<geist>
most of my x86s are now Zen based, so at least i dont feel as bad
<mrvn>
But I already go a PI into the work place working as power switch for out test systems.
<Bitweasil>
Yeah, my homeserver is AMD, my datacenter box is Intel.
<Bitweasil>
I couldn't get an AMD server I tried to behave. :( Not sure what was wrong with it, but it wouldn't stay running when idle.
<Bitweasil>
And I tried all the various BIOS setting hacks, disable deep sleep, etc.
<Bitweasil>
At some point, I'll probably either get a better pipe here and host locally, or add an ARM box into the DC.
<geist>
yah a bit worried the direction arm servers are going in: seems that most of them are going the vertically integrated route with hyperscalars
<geist>
since the individual producers of arm server cpus are having a tough time existing
<geist>
to ARM the company its all the same thing, they'll happilky sell you a arch license or a license to their cores
<Bitweasil>
Yeah. :(
<mrvn>
you still have to produce the silicon though
<Bitweasil>
I'd love to buy a server with 32 X2 cores or something and 128GB RAM.
<Bitweasil>
Just, not for $10k.
<geist>
but on that token ARM servers are definitely getting bigger
<geist>
so it's a net win for ARM itself
<Bitweasil>
Or, worse, "Oh, yeah, you can get a physical box in our cloud, $4k/month!"
<mrvn>
somebody needs to sell ARM to the cloud/webhoster people as security improvement, all those x86 exploits won't work.
<geist>
well they do. you can get ARM vms in a lot of places
<geist>
that's what i mean, cloud providers are doing it, but its generally in house
<mrvn>
not enough to drive the price down.
<geist>
price of what?
<mrvn>
boards I want to buy
<geist>
that's the point, they're not at all driving the price
<geist>
because they're not making physical cpus for sale
<mrvn>
yeah, the big hosters just order a charge of custom boards.
<geist>
no they *build their own*
<geist>
and that's fine to ARM, Ltd, because they make money either way
<mrvn>
don't think any are big enough to have their own PCB broduction and chip factory.
<mrvn>
s/any/many/
<geist>
oh of course they use TSMC or whatnot
<geist>
but taking the building blocks from ARM and building your own silicon is not that difficult, all in all
<geist>
if you are running a N billion dollar cloud business that's no big thing
<mrvn>
It's too bad TSMC can't just build 1% more boards and sell the left overs.
<geist>
anyway
<mrvn>
production runs have also gone down a lot in the last decade. you can just order 1000 boards.
<geist>
my point is there's very little market for an independent cpu vendor to make an ARM server and definitely a desktop chip
<Bitweasil>
Sadly. :(
<geist>
all the places it's really taking off have been vertically integrated
<Bitweasil>
And all the good stuff is high cost and going into phones anyway.
<geist>
ie company makes cpu for their own purpose
<Bitweasil>
Except Apple, but they're now evil.
<mrvn>
or worse a kernel dev board that has one of everything.
<Bitweasil>
"It has now been 0 days since Bitweasil mentioned disliking Apple." :p
<geist>
riscv will probably have a similar thing, is my guess. it soaks up the lower end space and highly in house custom stuff
<zid>
I dislike all tech companies
<zid>
I'm not a racist
<mrvn>
I hope the rock 5 can boot from SD.
<mrvn>
zid: do you hate racists and black people too?
<Bitweasil>
mrvn, I would *expect* something like a small SPI onboard that can then pivot to SD, eMMC, or NVMe.
<j`ey>
probably uboot
<mrvn>
Bitweasil: I've ordered the 50$ Discound code for a Rock5. Lets see what happens next.
<Bitweasil>
*nods* I did that too.
<Bitweasil>
I have 4 of the 16s on order.
<j`ey>
cluster :3
<mrvn>
Last time I ordered something without pre-VAT on it I had to drive 2 hours to pay 5EUR tax and pick it up. Hope this one will go better.
<mrvn>
Bitweasil: Do they need a heat sink+fan? Didn't even check.
<Bitweasil>
I don't know.
<Bitweasil>
They don't come with one, I'll evaluate it and see.
<mrvn>
That white connector on the left could be fan.
<Bitweasil>
Yeah, I'll play with them when they show up.
<mrvn>
The mSD card socket looks to be on the right under the WiFi slot. Where is the eMMC flash socket?
<Bitweasil>
Top left, far as I can tell. That's the style eMMC connector the N2+ and PineBook Pro have.
<Bitweasil>
Or... at least close.
<mrvn>
The 2x 2 gold lines? So no eMMC installed in the picture?
dormito has quit [Quit: WeeChat 3.3]
<Bitweasil>
Correct.
<mrvn>
And the 2 black boxes on the top right over the wifi? irda?
<mrvn>
or could that be poweron / reset buttons?
<Bitweasil>
I think so.
<mrvn>
But you can clearly see it's a quality product. It has gold contacts (for the power).
<immibis>
mrvn: why would they sell the extra 1% to you... instead of to their customer?
<mrvn>
immibis: 1) tghe customer only ordered N units, 2) small overproduction in case of defects
<mrvn>
promotion of their services by getting their products into more peoples hand
mahmutov has quit [Ping timeout: 256 seconds]
<immibis>
something something capitalism
attaboy has quit [Ping timeout: 240 seconds]
not_not has joined #osdev
<not_not>
Ahoy
<not_not>
Bitweasil cant u just crush the wifi?
<Bitweasil>
Once you've moved into "Someone has to desolder the wifi interface chip and verify that it no longer functions," it's cheaper to just get something that doesn't have wifi in the first place.
<not_not>
Oh
<mrvn>
I would cut the legs
<Bitweasil>
You may safely assume I care about a lot of weird edge cases that don't bother most people.
<not_not>
But if i make an lfs build compile all from source even check the compiler for hacks
<not_not>
Verify with checksums
<not_not>
I like u bitweasel
<not_not>
And flash the bios
<not_not>
Better yet build custom hardware
<not_not>
I designed a cpu in logisim wich has a seperate ram chip for iptrs so they dont lie on the stack for a teenager to overwrite
<not_not>
It ran brainfuck
<not_not>
I didnt wanna make a cisc, brainfuck is very risky
<not_not>
Waiting for clock to hit 0, then im not in danger of snoozing
dormito has joined #osdev
cyberbanjo has joined #osdev
cyberbanjo has quit [Remote host closed the connection]
heat has quit [Remote host closed the connection]
blockhead has joined #osdev
air has joined #osdev
kingoffrance has quit [Ping timeout: 240 seconds]
pretty_dumm_guy has quit [Quit: WeeChat 3.4]
jeaye has quit [Remote host closed the connection]