klange changed the topic of #osdev to: Operating System Development || Don't ask to ask---just ask! || For 3+ LoC, use a pastebin (for example https://gist.github.com/) || Stats + Old logs: http://osdev-logs.qzx.com New Logs: https://libera.irclog.whitequark.org/osdev || Visit https://wiki.osdev.org and https://forum.osdev.org || Books: https://wiki.osdev.org/Books
<klys> apparently 4addr requires another authentication mechanism, and there's an associated password
<klys> so today I ordered (not one but two!) usb ethernet dongles. I should see them in another week or so.
<bauen1> klys: tbh that sounds like something 1-2 bridge / veth interfaces should be able to accomplish, or how exactly are you gonna use those 2 adapters
buffet3 has joined #osdev
Belxjander has quit [Quit: AmigaOS PPC 4.1 +E +U1 // AmIRC 68K]
ckie has quit [Ping timeout: 256 seconds]
PyR3X has quit [Ping timeout: 256 seconds]
masoudd_ has quit [Read error: Connection reset by peer]
PyR3X has joined #osdev
buffet has quit [Ping timeout: 256 seconds]
buffet3 is now known as buffet
ckie has joined #osdev
<klys> bauen1, wire the laptop into itself, br0 is wlan0 and eth1, br1 is eth0 and tap0, tap1, etc.
<klys> enable proxyarp on wlan0 and we'll see how it turns out
ahlk has quit [Ping timeout: 250 seconds]
tomaw_ has joined #osdev
<klys> as far as i assignemts are concerned, br0 is my internal addy, br1 is 0.0.0.0 up, and the vm across from tap0 has an addy on the same network as br0. all other iface's I mentioned are 0.0.0.0 up
<klys> s/i assignments/ip assignments/
<mrvn> sonny: if you want performant networking you need zero-copy mechanics. If you do routing in software that is doubly true.
<klys> to try this experiment, one cannot simply clone a vm (and the corresponding mac address in the config). each vm will need a separate mac address (obviously).
<sonny> mrvn: I was thinking more about how it's slow to do networking through the kernel
tomaw has quit [Ping timeout: 606 seconds]
tomaw_ is now known as tomaw
<mrvn> klys: if you publish the your MAC for the IP of the guest on wifi and set up forwarding then you should route traffic transparently.
xenos1984 has joined #osdev
<klys> mrvn, you think qemu can do proxyarp still, though this feature has not been added yet
<mrvn> sonny: it's your job to not make it slow
<sonny> you'd need a configurable kernel that can switch interrupt modes or something maybe
<mrvn> klys: nothing to do with qemu, the IP layer in linux will forward traffic if you provide the ARP replies.
<klys> mrvn, try it yourself and get back to me. you'll need the parprouted package or instructions from https://wiki.debian.org/BridgeNetworkConnectionsProxyArp
<bslsk05> ​wiki.debian.org: BridgeNetworkConnectionsProxyArp - Debian Wiki
<mrvn> klys: I'm talking without any proxyarp magic. Hardcoded arp entries.
<klys> oh, does that take an arp command? here I'll read `man arp'
k8yun has quit [Quit: Leaving]
<mrvn> arp [-v] [-H type] [-i if] -s hostname hw_addr [netmask nm] pub
k8yun has joined #osdev
<mrvn> klys: you tell the WiFi that anyone looking for the IP of the guest should use the MAC of the host. Then the host gets the traffic, sees the IP is a mismatch and forwards it.
<mrvn> klys: On the guest you set the host as gateway so you don't have to add arp entries for the other direction.
<mrvn> n8
gog has quit [Quit: byee]
eroux has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
matrice64 has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
heat has joined #osdev
k8yun has quit [Read error: Connection reset by peer]
k8yun has joined #osdev
Terlisimo has quit [Quit: Connection reset by beer]
matrice64 has joined #osdev
Terlisimo has joined #osdev
nyah has quit [Ping timeout: 240 seconds]
k8yun has quit [Ping timeout: 272 seconds]
pretty_dumm_guy has quit [Quit: WeeChat 3.4]
k8yun has joined #osdev
<heat> i got tired of looking at the irq chip so now i'm trying to reach user-space
<heat> hooking up all the arch scheduling stuff
k8yun_ has joined #osdev
vin has quit [Ping timeout: 240 seconds]
k8yun has quit [Ping timeout: 272 seconds]
vin has joined #osdev
myon98 has joined #osdev
sonny has quit [Ping timeout: 256 seconds]
[_] has joined #osdev
[itchyjunk] has quit [Ping timeout: 250 seconds]
[_] is now known as [itchyjunk]
blockhead has joined #osdev
dh` has joined #osdev
dude12312414 has quit [Quit: THE RAM IS TOO DAMN HIGH]
sonny has joined #osdev
scaramanga has quit [Remote host closed the connection]
<geist> heat: this is riscv? having trouble with the plic?
<heat> kinda
<heat> the plic's design is weird and the device tree's interrupt stuff is weirder
<sonny> you can't design an OS without first having an idea of what the chip can do right?
<heat> if you're designing an OS around a chip it's not really an OS
<sonny> why not?
<heat> an OS is supposed to be platform/architecture independent
<kazinsal> dozens of platform-dependent operating systems would disagree
<sonny> oh
<heat> you can assume a minimum amount of stuff but assuming a platform or arch? nah
<sonny> so basically, so have to assume it's like a von neuman machine or something?
<heat> no lol
<sonny> e.g. virtual memory is a feature of quite a few processors now
<heat> assume its a CPU, it's X bits(you can assume 64, but 32 is also sane-ish), you can also assume an MMU
<heat> designing it as arch agnostic is a great idea
<sonny> but they had the hardware for it when they made the OS the first time
<heat> who?
<sonny> it's only common after
<sonny> multics
<heat> what does that have to do with anything?
<sonny> idk
<sonny> nevermind
<klys> multics didn't have an mmu
<heat> OS design has evolved
masoudd has joined #osdev
<heat> if you look at unix over the ages, it became progressively more portable
<heat> because portability is usually desirable
<heat> even if you think that your OS will only run on X
<heat> you lose pretty much nothing by making it portable, and you gain a lot
k8yun__ has joined #osdev
<klys> should I design a system for a chip with no mmu? I would still encourage doing so. paging exists, sure, why limit yourself honestly.
<kazinsal> sometimes you just gotta write an OS for a specific machine
<klys> so if you want to do that you can ask me about it
<sonny> klys: I'm just trying to figure out how to design one
<heat> i guarantee you designing a chip for a CPU without an mmu is stupid as hell
<heat> because all modern concepts of everything ever go out the window
<klys> sonny, design does imply some vision for what you want though
<heat> like, erm, fork or executables
<heat> s/chip/osdev/
<heat> erm, os
k8yun_ has quit [Ping timeout: 240 seconds]
<sonny> klys: well I know what an OS is, and what it's defined by creates the goals
<klange> The history of OS portability is interesting.
<heat> sonny, what do you want your os to do? that's probably the first step
<klys> sonny, an os is something you can read about in the a.tannenbaum book "modern operating systems." if you want a first-generation batch processing system, that avenue is avialable to you. if you feel like you need to expand on a first generation system to meet your principles, then there's a way, I'm sure of it.
heat has quit [Read error: Connection reset by peer]
heat has joined #osdev
<klange> when I started out, my answer to that question would have been "spite the uni board that decided a class I already took overlapped too much with the computer engineering department's OSes course"
k8yun__ has quit [Quit: Leaving]
<sonny> klys: I "read" the dino book, and richards unix implementation book
<sonny> there's a new free text called OSTEP but I think it's just unix stuff
<klange> Silberschatz, Pterson, Galvin? More of a history lesson than a book about operating system design.
<klys> yeah that is the dino book. and if what you want is to follow that outline, then there is still a way forward. you may not have 20/20 hindsight, yet the system will still operate one way or another.
<klange> Incredibly appropriate that its cover represented the OSes as dinosaurs.
<geist> yah i think that was the one i used in college
<sonny> an os abstracts hardware, the chip. provides encapsulation with time, and space (memory)
<sonny> the goal is of course to make such an os
<geist> i did like how silbershatz book has real world examples of various OSes
<klange> MOS is better, especially the updated editions, though it's still a rather high-level gloss of concepts. Tanenbaum reserves the nitty gritty for OSDI ('the minix book')
<klange> MOS does the same, but fitting with its title Tanenbaum talks about OSes that were more relevant to a modern audience [relative to the publication date; hence the updated editions are a better read, they'll cover more on Linux and NT than the first edition].
<sonny> MOS?
<klange> Modern Operating Systems
<heat> modern operating systems?
<klange> And OSDI is Operating System Design and Implementation
<klys> well sonny the thing is you need it to "boot." so, you get to start with a simple boot method, a beginning, and decide how to translate that bootloader process into a configuration. I think how you decided to reach your configuration is a solid step in the right direction.
<geist> frankly i think everyone should jsut read all of them, but that's a lot of time
<geist> and expense
<klange> er, insert 's: ' between System and Design.
<sonny> I saw some old ones, like design of CP/M and stuff but that seems to be covered by the dino book
<heat> you can also read none
<heat> that also works
<heat> it's less ideal but eh, you'll get there
<klange> I got my copy of MOS, first edition, when my high school APCS (and also calculus) teacher was emptying a closet.
<sonny> klange: I have been looking at linux and NT lol
<klange> The one with the dining philosphers illustrated on the front, the later ones depict a circus.
<sonny> klys: see, that's the thing, I wanted to start from the top down
<heat> that doesn't work
<klys> sonny, so have you got some design on the idea of a configuration?
<heat> you can't design one from the top down if you don't know how it works down there
<sonny> heat: you said it didn't matter lol
<heat> it's a whole process, at least the first time
<sonny> klys: what do you mean by configuration?
<heat> sonny, what doesn't matter?
<klys> sonny, when your operating system starts running, that process should be configurable.
<sonny> as in, select what to run?
<sonny> oh, how to set up
<sonny> heat: you said the low level details didn't matter
<klys> right, and when you reach that stage your bootloader process is clear. you've moved forward.
<heat> sonny: No I didn't. I said they should be abstracted in a decent enough level. I'm also not talking all the way down in that way. I'm saying you need to have an idea of how things actually work, and how the OS actually works in detail before designing something from the top down
<heat> and you probably won't have an idea of how the thing works down there, unless you've written it yourself, looked at lots of code or read some books
<sonny> I know some computer arch
<heat> if I asked you to design a whole virtual memory subsystem, could you do it? probably not
<sonny> the first thing is, why is it already a subsystem?
<klys> because you have an mmu
<heat> not only do you have an mmu, but you need a lot of code to interface with and manipulate the mmu
<heat> like, most code you'll have inside the OS doesn't depend on architecture X's mmu
<sonny> I did do a virtual memory thing, but it was for a weird OS ...
<geist> in the long run most of your code for an OS is arch-neutral
<heat> even most code inside the vm subsystem doesn't depend on the arch's MMU
<geist> when getting started its the other way, since you need to get the cpu up and running
<klange> it was really surprising to me, after I had cleaned things up and abstracted things better, how little I had to change for my aarch64 port
<geist> yah a proper abstraction nets that
<sonny> ok, so from that I get the impression "subsystems" are a required feature for an OS
<klys> sonny, so when you get to the configuration stage, you'll probably do some things like set up and take down. memory management is one of the things you can set up. no pressure here
<geist> also helps that from a fairly high level arm64 is pretty close to x86
<geist> same with riscv64
<geist> details are different, but the overall way it processes day is pretty much identical
<heat> sonny, that's not a feature, it's just a way to structure things
<heat> you'll have parts of the code that deal with VM, filesystems, networking, blah blah
<klange> It's like computer architecture has gone through an hourglass, looking back.
<heat> and those parts tend to be relatively big
<dmh> ive finally did it. shooting code into a loader over usb for painless baremetal arm dev
<sonny> I can't tell that from how you phrased it, and that's already part of the design
<dmh> sdcards around the world rejoice
<sonny> this is what I want to figure out
<heat> try to get something going first
<heat> write a simple thing (bootloader? kernel? your choice)
<heat> play around with your environment
<sonny> yeah, did that
<klys> so, virtual memory, filesystems, networking <- those are abstractions. the first two are data abstractions, and the third is the networking abstraction. this is where you get into some theory, like the seven layer model, etc.
<heat> sonny, keep going then
<heat> build on top of that
<sonny> klys: thanks
<heat> do you have something that boots and prints a hello world? see what else you can do
<sonny> so it looks like the core thing is an IO abstraction
<sonny> heat: yeah, that's what I did
<heat> then keep doing it
<sonny> and some drawing for the VGA
<heat> abstractions will come over time
<klange> The importance of building a 'hello world' is vastly underestimated and unappreciated.
<sonny> I can't keep going unless I know what I'm doing
<heat> why do you not know what you're doing?
<dmh> i just spent a week straight getting hello world going basically
<sonny> heat: cause I don't really understand
<klange> Almost all of my RPi400 platform work was just building a hello world, and then jumping from that hello world to my kernel.
<_eryjus> sonny, don't be afraid to try something and fail at it. If I had a nickle for every time I re-implemented my PMM.....
<klys> sonny, I'm here to help. applications, data, networks, systems, configurations, documents, and sources <- those are all the main tier abstractions in technology according to me.
<heat> sonny: what do you not understand?
<sonny> osdev, I don't actually get what's being made
<geist> klange: yah bare metal hello world when first fiddling with arch is the way to go
<heat> well, you're writing a kernel
<heat> that kernel lets user programs do cool stuff, manages the hardware for them, gives them CPU time
<klange> Ya need goals.
<heat> an OS is essentially a bunch of user programs that talk to a lower level program (the kernel)
<klange> Say, "I want my OS to play Doom". That's a goal. Once you have something that like that in mind you can figure out what you need to achieve that goal.
<klys> sonny, to have a systems level abstraction, you need all the other things too, of course. apply yourself. manage memory. build your network. standardize. document everything. and write legible source code.
<sonny> heat: well, that's already a descision right? for it to be a supervisor process
<heat> sonny, no, that's not a decision
<sonny> so it has to be?
<heat> it's a requirement
<sonny> hmm
<heat> all modern CPUs need something running in supervisor mode that does things
<sonny> sure, but after boot I wonder if it has to continue that way
<heat> even if that thing is *everything* (see DOS, lk, really tiny embedded kernels)
<sonny> oh, I thought DOS just didn't care
<heat> sonny, well, yes, how else would you do things?
<geist> that's because dos came into existance before the cpu had a notion of superisor mode
<geist> so dos just runs in whatever the highest priviledge mode was
<sonny> I see
<heat> if the CPU needs you to be in supervisor mode to change threads, interface with hardware, etc, how would you skip supervisor mode?
<geist> same with CP/M, etc. other single user/single tasking systems tend to do that
<geist> since there's nothing really to protect the system from
<sonny> so this is the first place where you need seperation
<sonny> and that's now a common feature, a la privelleged instructions
<heat> not only that, but the CPU separates kernel memory from user memory (through the MMU)
<geist> in general yes, as soon as you sdtart running multiple tasks, if you want them to be actually separated from each other (not a requirement) the you need to run the tasks in a lower priviledge mode (user) and the rest of the system in a higher priviledge. that's generally modern design
<dmh> only if ur a coward
<sonny> lol
<dmh> jk pls do pm :(
<klys> sonnt, dos runs executables. dos manages some memory. dos can use a packet driver network. dos boots to config.sys and autoexec.bat. dos is documented via reverse engineering largely. dos has available source code (just don't read that, write your own.)
<heat> process 1 can't steal process 2's memory if the kernel is handing it out (and is the only one that can hand out memory)
<sonny> I tried, but it's mostly asm code
<kingoffrance> imagine a test question a is to b as c is to ? . os/kernel:userland http://dict.org/bin/Dict?Form=Dict2&Database=devil&Query=magnet Something acted upon by magnetism. http://dict.org/bin/Dict?Form=Dict2&Database=devil&Query=magnetism Something acting upon a magnet.
<bslsk05> ​dict.org: dict.org- magnet
<bslsk05> ​dict.org: dict.org- magnetism
<sonny> I could probably read it now though
<geist> frankly if you want to actually write a dos clone, you will need to be able tor ead assembly
<geist> since that's the level you'll be operating at
<sonny> yeah I can do that
<geist> but writing a dos clone is not that interesting an exercise IMO. it's been done countless times
<geist> and at the end you're left with... a dos clone
<heat> it also skips all the fun stuff
<geist> yah
<geist> and is intrinsically unportable
<sonny> geist: I just wonder if there's classes of what an OS program is
<heat> in uni? yeah
<klys> yeah that's the abi
<sonny> it also must be bounded by the machine
<geist> i put it in the same category as other early 8 bit OSes and firmwares: apple2, atari 800s, c64. it's just a fancier version of that
<klys> there are distinguishable forms of programming that go with varying systems and that's called an abi
<sonny> ok so
<sonny> let me see
<klange> In the beginning, to program a computer, you flipped a bunch of switches, or got a bunch of skilled older ladies to knit a bunch of wire around magnetic donuts.
<heat> :D
<klange> Some time later, the idea of computers as general purpose instruments beget punch cards, and a computer operator was someone who stood a round shoving punch cards into the reader.
<sonny> ok, I have a better understanding I'm just trying to describe it
<klange> Operating systems are software that replaces the task of shoving the punch cards in the reader.
<klange> The punch card shover had to know which punch cards to shove in, and what order to punch them, and if they didn't work right they had to tell the person who provided the punch cards.
<klange> s/punch them/shove them/
zaquest has quit [Remote host closed the connection]
zaquest has joined #osdev
<sonny> So, the os provides abstraction for hardware. The design of the os program depends on the hardware. The abstractions can be split up to describe the key parts, such as IO and memory. You get security by defining boundaries, such as privelleged modes for IO? and memory right? Unix decided to solve IO by using files, so the killer abstraction for them was streams
<geist> basically yes
<geist> that is one of the main tasks a modern OS will do
<heat> streams?
<heat> XSI moment
<sonny> sorry, I meant sequences
<sonny> that's the math term
<klys> sequences of binary words are data
<sonny> and it seems like the more you require from the hardware, the more abstractions you will have to make?
<heat> in a general sense, yes I guess
<sonny> the difficulty being, you can't see what is required ahead of time
<heat> you don't need to
<heat> if you want networking, you add code to abstract networking
<heat> if you want wifi, you add code to abstract wifi
<heat> 3D drawing? same thing
<sonny> ok good
<sonny> and they key thing for being able to abstract hardware is the ability to work with memory addresses right? since that is sometimes how interfaces are described?
<heat> ?
<heat> here's a question: what do you need in order to load a user program and do "printf("Hello World!\n");"?
<heat> try to work that out
<sonny> well, nothing
<heat> that's actually a great objective too
<heat> nothing?
<sonny> I can just get the bytes can't I
<klange> where are you going to get them from?
<sonny> the cpu will run my instructions
<heat> how would you load the user program? also, what user program? how does printf do the printing?
dmh has quit [Quit: ZNC - https://znc.in]
<sonny> klange: I could use something electrical to send signals
<heat> don't be so abstract
<sonny> it's the only way I can understand
<geist> it's not a helpful level of abstraction though
<sonny> I am going to do more electronics stuff so it makes sense
<heat> something electrical to send signals is helpful if you're making your own CPU but incredibly unhelpful if you're building an OS
<sonny> geist: they used to have switches for these on old computers
<sonny> the first firmware
<sonny> or so I was told
<geist> sure
<heat> sonny, where is your IRC client stored?
<heat> or browser, same concept
<klange> Yes, and the operating system in that case was the human being flipping the switches. (Not to be confused with the programmer, who was the one who told them which switches to flip)
<sonny> not that old, but I get it
<sonny> there was some code you had to load manually, guess it was the same process
<sonny> heat: well, in memory
<heat> well yes, but where did it come from?
<sonny> source code that was turned into an executable
<heat> your DDR4 ram sticks didn't come with chrome installed
<sonny> lol
<klys> memory addresses come out of the system, address data, form sequences, and collate input. data, otoh., goes into your compiler, your outputs, your files, and even your operations, your hardware, and sometimes your physical media. if memory addresses are a key to anything, they're a key to your applications, data, and network. memory addresses are not data, though they are the key by which data is addressed.
<heat> sonny: do you have a hard drive? or an ssd?
<sonny> ssd
<heat> well, your chrome is stored on the ssd right?
<heat> or $favourite_browser
<sonny> yeah (sounds like a trick question)
<heat> no
<heat> it's an honest question
<geist> just trying to get you to think through the details
<sonny> ok
<heat> in order to get the bytes for chrome, what does your OS do?
<klange> Let's say you're already running. Someone already flipped the switches to load _you_, they don't want to flip more switches - they expect you to do the rest.
<heat> "get electrical signals from the SSD" is the wrong answer btw, that's not really the question here ;)
<sonny> well it knows where chrome is stored
<heat> right
<heat> the OS finds out where chrome is stored
<heat> and **asks** the SSD to read chrome
<heat> makes sense right?
<sonny> yeah
<heat> the SSD does the electrical magic and you, the OS, get the bytes
<heat> that's also the case for my tiny hello world program
<heat> so, the OS loads the program (think: virtual memory, scheduler)
<heat> how does printf() write something to the screen?
<sonny> asks the screen if it can write
<heat> well no, the user program can't talk directly to the GPU
<klys> screening applications have changed, and asking the graphics hardware if it can write: is probably a more accurate answer today than it was 30 years ago. then, the display would refresh every so often (in Hz), and you could just plant whatever you wanted there and it would be visible coming out of the ramdac.
<heat> (think: the kernel manages hardware, it also usually manages other concepts, like terminals)
<sonny> yeah, so it has to ask?
<heat> yes
<klange> There's this classic software engineering interview problem I used to really like, it was even a focus of my OSdev work for a while. It's deceptively simple:
<heat> it calls the kernel(concept: system call) and tells it to write "Hello World!\n" to the terminal
<klange> What happens when you type 'google.com' in your web browser's address bar and press enter?
<sonny> sends a web request for google's home page
<klange> You can answer at a very high level and say the browser sends a request to google, google responds, and the browser renders it.
<klange> But you can answer at increasingly deeper levels of detail.
<heat> what's a web request, what is 'google', what is 'home page'
<klys> first it loads http://google.com/index.html which is a redirect to https://www.google.com/ and so encryption ensues, cookies are gathered, etc. more stuff further on down the line
<bslsk05> ​google.com: Google
<klange> Any answer in which requesting a document is the "first" thing that happens is so far away from being sufficient from the perspective of implementing an operating system.
<heat> sonny: you can look at the Q&A about the tiny hello world program and that pretty much fills your schedule for the first year
<sonny> yeah, I saw the hello world for freebsd
<sonny> it was epic
<heat> like, how does the OS know where chrome is on the SSD? what's an SSD? Is there an SSD? How do I talk to it?
<heat> what's a program? what's a process? where is memory? how do I load a program?
<klange> What is a key press? How does your browser even know you "pressed a key"?
<sonny> interrupts
<heat> great answer! but how do you get interrupts? :/
<sonny> they are implemented in the processor
<heat> yes and no
<sonny> if you want to use them, you send a signal
<heat> thanks electrical engineer sonny! but that doesn't help software engineer sonny
<klys> the programmable interrupt controller was integrated into the first soc about when we stopped using those northbridge/southbridge chips
<geist> true, though that's a highly x86 centric way of viewing it
<heat> SWE sonny wants to know how he knows there's an interrupt, how to differentiate interrupts (most hardware can send interrupts), how to respond to an interrupt
<geist> one can argue, for example, that 68k had one from day one. and obviously older cpus/minis/etc had various amounts of interrupt controllers
<sonny> heat: so I have to make an abstraction?
<heat> what do you mean?
<klange> We're not necessarily expecting you to know the answers to these questions. But figuring them out, and understanding that there are steps between "finger pushes key" and "browser requests page" and understanding which steps in that are your responsibility as the operating system ... is the goal here.
<sonny> to let the software guy know what it is
<klys> you have to make a configuration so your abstractions will form a design
<sonny> geist: trap instructions are pretty old
<heat> sonny, SWE sonny knows what an interrupt is. but he needs to find out how to know what hardware wants his atention, how to respond to the interrupt
<sonny> I see
<heat> some things can change from cpu to cpu (how to respond to the interrupt, how to ack the interrupt), but most things dont (what is an interrupt? what hardware wanted my atention?)
Jari-- has joined #osdev
<heat> the first part goes in arch/x86, the second goes in interrupt.c
<sonny> well your cpu doesn't require them, but yeah
<Jari--> morning
<sonny> gm
<klys> hi jari--
<heat> sonny, most CPUs do require interrupts though
<sonny> thanks, I have a much better understanding now
<heat> np :)
<heat> going for interrupts is actually a great first step
<sonny> the key is the heirachy of abstractions
sdfgsdfg has quit [Quit: ayo yoyo ayo yoyo hololo, hololo.]
<heat> yes. it's also about making useful abstractions
<sonny> what is portable is the highest layer, the one to present to the programmer
<klys> and this is why we want portable applications
<heat> your SSD driver doesn't need to know how interrupts work in the current mobo/CPU, it just needs to know how they work and how to receive one
<heat> it needs to know *that they work* sorry about that
<heat> related: what happens if your SSD driver does something illegal to the eyes of the CPU? (concept: traps/CPU exceptions)
<sonny> hmm
<sonny> I was reading the virtualbox manual, they have a bunch of these interfaces already
<heat> getting these things done are great first steps
<sonny> you could build an OS of that right?
<heat> what interfaces?
<heat> how does virtualbox help you here?
<sonny> virtualbox is an x86-64 emulator
<heat> i know what it is
<heat> it emulates the machine itself, but you're writing software
<heat> you can interface with the machine, for sure
ElectronApps has joined #osdev
<sonny> I thought they seemed to describe devices and stuff in an api https://www.virtualbox.org/sdkref/annotated.html but if that means something else then nevermind
<bslsk05> ​www.virtualbox.org: VirtualBox Main API: Class List
<sonny> otherwise yeah I'd have to do it from scratch
<heat> well no, that's their API for the actual virtual box program
<heat> i suggest you get an objective, like the "load a hello world user program and run it" we went through
<heat> for every answer of every question you went through, try to flesh the concept out, see how it works for your architecture (I assume x86 since you mentioned VGA) and do it
<sonny> x86 was what I used yeah
<sonny> well, the 64 bit version
<heat> when you flesh everything out, you'll have a loaded hello world program that does the beep boop
<heat> congrats
<heat> you can't really design an os from the top down (since you don't know a lot of details), but you can design your way up, by picking something you want to do, and finding out what you need in order to do it
<heat> user program -> filesystem -> disk -> memory, interrupts -> cpu specific code that deals with the mmu, interrupts
eroux has joined #osdev
<Griwes> the worst part of writing an OS is where you remove the last commented out lines of your current "main" function and just stare into the void of trying to figure out the next goal again
<klys> user mode scheduling
<heat> run chromium
<klange> "play doom" is always a good one
<sonny> heh
<sonny> won't I have to port those programs?
<heat> yes
<sonny> fun
<heat> if chromium wants to access the internet you need to add that, if doom wants to play audio, you need to add that
<heat> the fun of osdev is that you need to do most things yourself
<heat> no nodejs module can help you here
<heat> (well, sometimes you can use third party projects, but that's not the point)
<sonny> so, tell me, why does no mmu suck?
<klange> Porting something that already exists is a great goal. You can come up with all sorts of workarounds and shortcuts and cheats if you're just getting your own code to run...
<sonny> it should only be horrible if your OS support virtual memory right?
<sonny> or, isn't configurable rather
<heat> most modern OSes want to protect processes from messing with each other
<sonny> yeah
<heat> it's the whole basis of of security
<sonny> so that's it?
<heat> nommu means anyone can touch anything
<sonny> hm, ok then
<heat> it also means that, for example, if your hello world program was built to run on address 0, nothing else can run on address 0
<sonny> yeah
<heat> unless you try to fix it (linux has), but it's messy and way more complicated
sdfgsdfg has joined #osdev
<sonny> do you know of any os that can handle no mmu, and virtual memory?
<heat> modern (general purpose) operating systems have worked on top of MMUs for so long that lots of things don't really fit on top of nommu
<heat> linux for example
<heat> probably some of the BSDs
<sonny> those require mmu
<heat> no, linux doesn't
<heat> I bet netbsd doesn't as well, because that thing runs everywhere
<sonny> for some reason I recall seeing that in kernel docs somewhere
<sonny> I think the netbsd folks said they require and mmu as well, dunno about the rest
<geist> i think there are nommu builds of liux, but not sure that's in mainline or not
<geist> or is a branch of
<heat> geist wrote a kernel that can run on nommu and mmu
<sonny> oh ok
<sonny> geist: did it have a mode?
<geist> and yeah pretty much anythig above embedded or whatnot requires mmu nowadays
<geist> more specifically *paging* mmus
<geist> vs something more primitive like segmentation
<sonny> or is it, the nommu design also works on mmu?
<heat> the only big difference you have is inside the "vm" subsystem
<heat> if you call mmap(2) in user-space, something will have to give you memory
<geist> sonny: you mean the LK stuff. can run in nommu on some arches. but it depends on if there's code to suppor tit
<geist> i only really allow that on riscv since those come in all flavors
<sonny> ok
sdfgsdfg has quit [Read error: Connection reset by peer]
<geist> and i think clever hacked in some nommu support on the arm64 path
<sonny> oh neat, riscv
<heat> if its no mmu, you just allocate a bunch of memory; if it has mmu, you need to allocate a bunch of memory, and map it by using the CPU's mmu
<geist> it gets hard to abstract as the VM/NOVM interface gets more complicated
<heat> riscv is also a really solid choice if you want something simpler
<geist> right and in the no mmu case you hav emuch less options *where* you allocate
<geist> because you only have all of the physical memory to carve up
<geist> but tha's basically what the novm code in LK does. it has a simpler bitmap based pmm that lets you grab runs of physical and then it acts like it 'mapped' it for you
<heat> and that directly conflicts with things like fork(2), that literally don't work in linux nommu
<heat> how do you duplicate an address space without an mmu?
<sonny> you don't
<heat> exactly
<klange> with a disk ;)
<heat> fork() doesn't work in nommu
<klys> you can get a riscv dev board that runs uclinux embedded in 32gb with serial output or you can get a more advanced one that runs debian in a gb, has ethernet, video, and usb. both solutions boot from microsd.
<klys> s/32gb/32mb/
<heat> unless you page everything to disk a-la unix
<geist> yep. or memcpy it out of the way on context switch
<heat> ^^or you can use qemu and not go broke for riscv
<klange> that's how fork worked in classic unix; parent process was the one written to disk, memory was left as is, execution continued as the child
<sonny> yeah, I have to see, qemu may not be as available on windows
<geist> original unix model was more of a segmentation based system. built around the hardware it had
<geist> but siilar in concept: a segment maps runs of virtual address X - Y to physical address W-Z
<sonny> klange: haha
<sonny> unreal
<heat> modern linux tackles the process creation on nommu problem by simply faking fork() for process creation
<geist> and the kernel carves up physical space into runs of physical ram, assigned dynamically to segments
<geist> and in original fork it could literally just stop and copy the segment to another location in physical and then you have forked it
<klange> It's far easier to write an OS that uses virtual memory on a system with a real MMU. A lot of mistaken beginners ask "do I really need virtual memory?" thinking it would be easier to _not_ have it, but this is very much not the case.
<heat> you call vfork(), and vfork() suspends the parent process and the child only has a single chance of calling exec of another process, everything else is totally undefined
<heat> once you exec, the parent resumes
<sonny> given the class of devices that were mentioned, they all have mmu so you might as well
<geist> sonny: qemu is totally available on windows
xenos1984 has quit [Remote host closed the connection]
[itchyjunk] has quit [Read error: Connection reset by peer]
<sonny> I guess via cygwin
<heat> no pls no no cygwin god no
<sonny> lol
<klys> you could compile it with msys3
<klange> No, it's a native build.
<klys> you could compile it with msys2
<klys> imean
<heat> there's a native build
xenos1984 has joined #osdev
<heat> you can also run it inside WSL (highly recommend)
<klange> https://qemu.weilnetz.de/w64/ these are the "official" QEMU Windows builds
<bslsk05> ​qemu.weilnetz.de: QEMU for Windows – Installers (64 bit)
<heat> recent W10/11 WSL even lets you have linux create windows inside windows
<heat> window-ception
<sonny> thanks
<sonny> WSL is kinda frustrating, but that's just in my head
<heat> I also can't stress enough how broken doing osdev on windows is
<klange> heat: it's a whacky Wayland over RDP thing, I had a better experience with Xming
<heat> WSL is a godsend
<sonny> heat: llvm toolchain is available
sdfgsdfg has joined #osdev
<sonny> the main thing is just going to be the emulator
<heat> you can use llvm or gcc, but then you probably want to use GNU tools and GNU make, so you need that and eventually, you're using msys or cygwin
<klys> well i wouldn't be so sure the official qemu release supports riscv64 targets in the windows installer
<heat> and those are just worse
<klys> I don't see anything about riscv in the attached changelogs
<sonny> I will do riscv later, I'm almost done my arm book
<sonny> just have to do parallel processing
<heat> you can totally skip any sort of visual output anyway
<heat> everything can output to serial
<heat> you just won't be able to port doom to it ;)
<heat> unless you write something that does opengl -> ASCII art :P
<sonny> can I simulate an lcd device? lol
<klange> heat: ooh i have a thing for this
<heat> you what
<klange> one moment
<klys> the allwinner d1 has an optional lcd, the driver is here: https://github.com/rvboards/linux_kernel_for_d1/tree/rvboards_d1/drivers/gpu/drm/sun4i
<bslsk05> ​github.com: linux_kernel_for_d1/drivers/gpu/drm/sun4i at rvboards_d1 · rvboards/linux_kernel_for_d1 · GitHub
<heat> ok klys but that's not the point
<sonny> that's ascii?
<heat> let me guess this is running on xterm?
<klange> xfce-terminal
<sonny> wow
<klange> it's just 'pure' half-block and color escapes
<heat> damn
<klange> i also have a reduced color palette mode, and scaling options
<heat> how slow is it?
<klange> locally? entirely playable, if you can excuse the horrible input because in 2022 we still don't have even a prototype of a direct input spec for ttys
<klange> over ssh? really dependent on latency to server
<sonny> hmm
<klange> I cooked this up last September after someone joked about Doom over telnet.
<klange> Really the input is a far bigger problem than the display.
<heat> this is so sick and I love it
<masoudd> can dithering help with the colors
<masoudd> ?
<klange> Don't really have the pixels for it.
<klange> The full-res screenshot is massively zoomed out, that's a _lot_ of "text" being pushed.
<klange> this is nearest-neighbor downscaling, normal 80x25 terminal, works in palettized 256-color mode just fine
<klange> this is downsampled with 24-bit color
<klange> not exactly production-quality code, but it's over here if ya want to poke it further: https://github.com/klange/doomgeneric/commit/d20709732a66067c866dd5d90d0e1e873c88632c
<bslsk05> ​github.com: Doom in your terminal · klange/doomgeneric@d207097 · GitHub
<klange> To get the input to not be terrible, turn off key repeat delay, ramp key repeat rate to the extent of your ability to still type, and note that you can't press two keys at once
sonny has quit [Remote host closed the connection]
ravan has joined #osdev
the_lanetly_052_ has joined #osdev
ZipCPU has quit [Quit: ZNC 1.7.5+deb4 - https://znc.in]
chronon has joined #osdev
ZipCPU has joined #osdev
ravish0007_ has joined #osdev
ravish0007 has quit [Ping timeout: 240 seconds]
ravan has quit [Read error: Connection reset by peer]
ravan has joined #osdev
sdfgsdfg has quit [Quit: ayo yoyo ayo yoyo hololo, hololo.]
<klange> ... does the RPi400 not have an RTC? Huh. I mean, historically that makes sense, but I'm surprised the 400 didn't get one... guess they're still expecting NTP on boot.
<heat> no, it doesn't
<heat> no rpi has a clock
<heat> no rpi even has flash except rpi 4 onwards
<heat> and when you're accessing it through spi you can actually hear it, it interferes with audio
<klange> lol
heat has quit [Read error: Connection reset by peer]
heat has joined #osdev
dmh has joined #osdev
dmh has quit [Quit: rip]
ravan has quit [Ping timeout: 256 seconds]
wolfshappen has joined #osdev
dmh has joined #osdev
wolfshappen has quit [Client Quit]
dmh has quit [Quit: rip]
GeDaMo has joined #osdev
wolfshappen has joined #osdev
dmh has joined #osdev
dmh has quit [Quit: rip]
dmh has joined #osdev
Belxjander has joined #osdev
<bradd> with my gcc cross compiler, should I be using -mcmodel=large for the kernel and drivers?
<Mutabah> usually, you'd use `kernel`
<klange> I think it's important to know what each model does/allows for. 'kernel' lets the codegen and linking assume everything that is absolutely addressed is going to reside together in the -2GiB region.
<bradd> when I use kernel, I get a 'does not support PIC mode'. I plan to load my kernel from uefi from a .o file
<klange> 'large' says "don't assume that any relative offset is going to be that nearby"
<bradd> so if my kernel is > 2gb, something at the start may not be able to call something past offset 2gb?
<heat> -fPIC is totally different
<heat> afaik you can load yourself anywhere (but it's slightly less efficient and you need to relocate yourself)
<klange> It's more if you load the kernel in one place and the drivers somewhere else entirely, you may not have the space necessary in the relocations to actually fill in the
<klange> offsets
<bradd> ok. I think I'll use the =large model then for kernelspace.
<heat> no
<heat> use =kernel or (nothing + fPIC/fPIE)
* klange uses large on x86, but only because some of silly legacy stuff
<moon-child> >2gb kernel sounds _quite_ farfetched
<bradd> yeah, it does
<bradd> lol
<klange> it's not so much about size, but sparseness and loading drivers separately
<moon-child> sure
<bradd> ok. I guess I can load the kernel, then load the drivers somewhere else and just call the drivers entry point(probably the offset that the driver was loaded to)?
<bradd> and maybe pass a pointer to the driver with an array of function pointers into the kernel for driver-kernel communication?
<heat> drivers are commonly part of the kernel
<heat> you can also have modules but that's less trivial and isn't something you should be worrying about as a beginner
<bradd> ok. I'll write my elf loader to process only *_32 stuff then
<heat> _32?
<bradd> (for now)
<bradd> like R_X86_64_PC32 and friends
<heat> ah yeah
<heat> you'll find plenty of those in PIC
<bradd> ok, thanks for the help. I have a direction now
<klange> My kernel module loader does _64, _32, and _PC32.
<heat> np
<klange> I did actually fix this with my aarch64 builds... platform shim sets up MMU and loads kernel at -2GiB, kernel doesn't really care much about where it is physically, everything works nicely...
sdfgsdfg has joined #osdev
Jari-- has quit [Ping timeout: 272 seconds]
gog has joined #osdev
lkurusa has quit [Quit: I probably fell asleep (or went out). Who will ever know.]
pounce has quit [Ping timeout: 240 seconds]
dormito has quit [Quit: WeeChat 3.3]
kalla has joined #osdev
<heat> ok woohoo i can execute userspace code
<g1n> congrats heat!
<heat> thanks
<heat> it's really half assed but it's something
<kalla> I have fairly simple booloader which just sets up the GDT,enters protected mode and starts pulling the kernel from the hard disk and jumps to it. I dont see any issue when this is running on qemu, but on VMware Workstation my VM just crashes on the instruction "rep insw", edi value is 0x0100000 , es/ds/ss is 0x0. I see it crashing when i do "mov WORD [edi] ,0x41" as well. So i think this has got something to do with the address 0x0100000. Any hints on why
<kalla> this could be happening?
<GeDaMo> What about cx?
<heat> kalla, do you enable the a20 line?
<geist> heat: oh woot
<geist> re user space. once you get excpetions working it's pretty much no big thank for riscv
<geist> side note: i preordered one of those starfive riscv boards, so when that ships will be able to give a report
<heat> i had to wrestle with sstatus
<heat> it's awkwardly different from x86
<heat> tracked down a bug in platform_yield() where I made a fake exception but didn't really make it correctly
<heat> i.e interrupt flag was set but not the "interrupts were enabled when trapped" flag, and then I wasn't setting the "hey we came from supervisor mode" flag
<heat> i was also swapping tp's in a wrong way
<kazinsal> aw dang, my vax is going to be delayed a week because they had to replace the RTC battery as it was getting dangerous
<heat> and then I wasn't swapping stacks at the right place
<heat> it was a bit messy
<kazinsal> nice of them to inspect it and go "oh shit that's not good" though, saves me from having to order and then solder in a new battery
<geist> yah
<geist> now that i think about it i dont remember what the state of my battery is
<geist> i thought i swapped it, but maybe i didnt?
<kazinsal> so new ETA is the week of the 7th instead of the week before
<kazinsal> which I guess is alright since I'm on vacation that week
<kazinsal> (dumb bad VAX osdev stuff? I think so)
<geist> b oo
<geist> oh side note, in case you dont wanna futz with it i have a prebuilt vax toolchain for linux
<geist> took some hackery and whatnot so not easy to reproduce
<kazinsal> I've got an AUI transceiver on the way too so I'll be able to hook it up to the 'net
<heat> build 4.4bsd for it
<kazinsal> oh nice
<kazinsal> that'll be handy
<geist> yeah but this one is modern and supports modern language features, etc
<geist> one of the challenging things though is that vax doesn't support the full suite of modern atomics
<heat> but doesn't support K&R C
<heat> so you can't build retro code
<kazinsal> yeah, that's perfect. I bet I could write an ethernet driver for whatever early LANCE the vax is using and have a modern internet router running on a vax
<geist> yah
<geist> t'would rule
<kazinsal> as far as disks go I'll probably set up the scsi2sd to emulate a drive for VAX, a drive for BSD, and a drive for osdev hackery
<geist> i think there are slots in it so may be able to scrounge up a second card for it?
<kazinsal> yeah, it's two internal slots
<kazinsal> and four of the microvax/vaxstation peripheral slots on the back
<geist> i haven't looked into what kinda cards go in there
<kazinsal> you can get some neat stuff
<kazinsal> another microvax listing of the same model but with 16 megs of memory instead of 32 had a 8-port UART card in it but cost twice as much as the 32 meg one with no cards
<geist> does yours have a tape drive? mine does but no real luck finding media
<geist> so generally a waste of time
<geist> heat: there's a nice 4.3 bsd build that lots of folks run on vaxen. 4.3 BSD quasijarus
<geist> it's a nice compromise between modern bsd and period
<geist> for netbsd i found that 3.1 was a pretty good one (or maybe 3.0). reasonable speed, pretty stable, but resonably modern features
vdamewood has quit [Remote host closed the connection]
<heat> geist: two q's: how do I know what extensions the CPU supports and do you know what the default rounding mode should be?
<geist> misa has the features
<geist> and rounding i dunno
<geist> and if you're in supervisor mode you can't read misa so i think it's up to the device tree
<heat> all I want in life is a k&r compiler so I can port 4.4bsd to x86 or something
<geist> maybe a field in the cpu tree?
<heat> ohh
<heat> I think it's an sbi call
vdamewood has joined #osdev
<heat> actually no
<heat> huh
<heat> riscv,isa = "rv64imafdcsu"; <-- so I need to parse this crap?
<geist> yep. the misa register is basically that too
<heat> jeez
<heat> how does userspace get this info?
<geist> up to the OS. liux does it via an aux[] value
<geist> same with ARM. ARM cannot read cpu features in EL0
<heat> ah
<geist> riscv not exposing misa to even supervisor mode is anothe rone of those 'makes it easier on virtualization hardware' features
<geist> as in that's one less thing that would probably have to be trapped and emulated in a virtualization evironment
<geist> re: that rv64... string. there's a method to that madness. each of the letters is a feature
<heat> i know
<heat> but it feels stupid to parse a string
<heat> this is linux libc levels of madness
<geist> fun thing: the misa register is a bitmap of 26 letters
<heat> oh wow
<heat> cute
<geist> yah, clearly that scheme will run out of runway pretty soon if it hasn't already
<geist> there are already 'Z' extensions which dont fit in the alphabet
<wootehfoot> geist, naive question, can function-multiversioning somehow workaround parsing? https://lwn.net/Articles/691932/
<bslsk05> ​lwn.net: Function multi-versioning in GCC 6 [LWN.net]
<heat> Tentatively reserved for Dynamically Translated Languages extension???????
<heat> wootehfoot, no
<heat> glibc parses whatever linux gives it and does the thing
<heat> you always have to parse something
<wootehfoot> heat, allright, so function-multiversioning defers the parsing to glibc, good to know, thanks :)
<heat> yea
* geist sleeps
<heat> i think most other libcs don't support it
<heat> yea i should sleep too
<heat> i'm kinda deaded
heat has quit [Ping timeout: 240 seconds]
<kalla> heat, yes.. i have enabled A20 line
<GeDaMo> kalla: heat left
<GeDaMo> What form does your crash take?
dennis95 has joined #osdev
<zid> praying mantis
<mrvn> wootehfoot: The article says function-multiversioning uses cpuid and I assume it means the opcode there
vdamewood has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
sdfgsdfg has quit [Quit: ayo yoyo ayo yoyo hololo, hololo.]
<wootehfoot> mrvn, good point, not entirely sure, but I agree: since it's an arch (-march=rv64imafdc -mabi=lp64d), and not ISA subset (like ie. AVX), function multi-versioning feel applicable here
<wootehfoot> doesn't feel applicable here*
xenos1984 has quit [Remote host closed the connection]
xenos1984 has joined #osdev
<mrvn> wootehfoot: nah, should be fine. The article says you can use anything -march accepts. Try it and see.
nyah has joined #osdev
<mrvn> hehe, 404 for math geeks: https://mathsgear.co.uk/products/braino
<wootehfoot> haha, it gave you the Ø hehe, that's so dry..
masoudd has quit [Quit: Leaving]
kalla has quit [Quit: Leaving]
<mrvn> It's always amazing when you see extra checks in code to fix broken input like "no ']' found, We will just tack one on the end" and then nobody ever tested that case it seems: input 'foo[1-3' ---> output 'foo]'. Oh wait, while parsing we replaced '[' with 0 so adding ']' goes to the wrong place.
blockhead has quit []
terminalpusher has joined #osdev
<mrvn> if (!(orig = str = strdup(hostlist))) {
<mrvn> Who writes such code? What does that even do? *think* *smoke coming out of my ears*
<GeDaMo> Copies a string and gives you a pointer to iterate with?
<zid> It's odd, but still pretty readable
[itchyjunk] has joined #osdev
<mrvn> yep, orig is needed fopr free() at the end and str is used to iterate
<mrvn> and checks if the malloc fails
lkurusa has joined #osdev
lkurusa has quit [Client Quit]
<mjg> str could be assigned in the for loop or whatever
<mjg> #accountingfortaste
<mrvn> mjg: it gets passwd to a tokenizer function
<mrvn> while ((tok = _next_tok(sep, &str)) != NULL)
<mrvn> I like my code to do just one thing at a time. I also like comparing pointers to NULL/nullptr instead of the implicit conversion to bool, but that might just be me.
ElectronApps has quit [Remote host closed the connection]
[itchyjunk] has quit [Remote host closed the connection]
masoudd has joined #osdev
[itchyjunk] has joined #osdev
[itchyjunk] has quit [Remote host closed the connection]
[itchyjunk] has joined #osdev
X-Scale` has joined #osdev
X-Scale has quit [Ping timeout: 252 seconds]
X-Scale` is now known as X-Scale
lkurusa has joined #osdev
terminalpusher has quit [Remote host closed the connection]
X-Scale` has joined #osdev
X-Scale has quit [Ping timeout: 252 seconds]
X-Scale` is now known as X-Scale
lkurusa has quit [Quit: I probably fell asleep (or went out). Who will ever know.]
pretty_dumm_guy has joined #osdev
ravan has joined #osdev
nyah has quit [Quit: leaving]
mahmutov has joined #osdev
sdfgsdfg has joined #osdev
sonny has joined #osdev
sonny has quit [Ping timeout: 256 seconds]
the_lanetly_052 has joined #osdev
skipwich has quit [Remote host closed the connection]
skipwich has joined #osdev
the_lanetly_052_ has quit [Ping timeout: 252 seconds]
skipwich has quit [Ping timeout: 256 seconds]
tomaw has quit [Quit: Quitting]
sonny has joined #osdev
tomaw has joined #osdev
skipwich has joined #osdev
pretty_dumm_guy has quit [Ping timeout: 272 seconds]
dennis95 has quit [Quit: Leaving]
pretty_dumm_guy has joined #osdev
pretty_dumm_guy has quit [Client Quit]
pretty_dumm_guy has joined #osdev
pretty_dumm_guy has quit [Client Quit]
pretty_dumm_guy has joined #osdev
ravan has quit [Ping timeout: 272 seconds]
the_lanetly_052 has quit [Ping timeout: 272 seconds]
sonny has quit [Remote host closed the connection]
sonny has joined #osdev
dennis95 has joined #osdev
sonny has quit [Remote host closed the connection]
sonny has joined #osdev
sonny has left #osdev [#osdev]
mahmutov has quit [Ping timeout: 272 seconds]
mahmutov has joined #osdev
<mrvn> Now I wonder if multi-versioning (https://lwn.net/Articles/691932/) could be done to handle some ARMv6-early, ARMv6-late, ARMv7, ARMv8 in 32bit mode, differences.
<bslsk05> ​lwn.net: Function multi-versioning in GCC 6 [LWN.net]
<mrvn> How hard would it be to reimplement that dynamic loader step.
nomagno has quit [Killed (NickServ (GHOST command used by nomagno_!~nomagno@193.31.69.185))]
sdfgsdfg has quit [Remote host closed the connection]
GeDaMo has quit [Remote host closed the connection]
dude12312414 has joined #osdev
dude12312414 has quit [Remote host closed the connection]
heat has joined #osdev
<friedy10-> Is there a way I can forbid linux from accessing a certian physical address range?
<heat> i think you can poison pages
<heat> not sure tho
<clever> there is a bad-mem thing in the kernel cmdline, to ban chunks of ram
<friedy10-> Is there something I can do in the kernel configuration?
<heat> probably not
<zid> You could edit your bios's e820 code ;)
SanchayanMaity has quit [Ping timeout: 240 seconds]
paulbarker has quit [Ping timeout: 240 seconds]
paulbarker has joined #osdev
SanchayanMaity has joined #osdev
xenos1984 has quit [Remote host closed the connection]
xenos1984 has joined #osdev
dennis95 has quit [Quit: Leaving]
thaumavorio has joined #osdev
dormito has joined #osdev
<mrvn> What do you think about the RPi4 (beta) bootloader including a network installer?
<clever> mrvn: it makes some open firmware things simpler
<bslsk05> ​imgur.com: Imgur: The magic of the Internet
<clever> basically, bootcode.bin has a hard 128kb size limit, due to hw limitations
<clever> adding https and the usb-keyboard interface made it too big
<clever> RPF solved that by cutting the binary in half, the new bootcode.bin only does dram init and loading of bootmain
<clever> bootmain then handles all of the old stuff (sd, usb, tftp, nvme boot), plus the new https and HID stuff
<mrvn> Well, a 128kb image just means we have to optimize the code for size :)
<clever> the new bootmain lacks dram init, and is over 200kb
<clever> that implies they where at least 80kb over the limit
<clever> but, that bootmain is basically the same format as start4.elf, which i can already make
<mrvn> Is the network install stuff coded for the VC or does it load an ARM kernel?
<clever> so i could replace bootmain now, and not have to deal with dram init
<clever> its still codec for VC
<clever> the network-install downloads a ~22mb boot.img file, which is a fat32 disk image
<clever> inside that, is start4.elf, kernel.img, and an initrd
<clever> the firmware then boots from that firmware ramdisk
<mrvn> yeah, saw that. Obviously the flash application is a lot bigger than 128k.
<clever> the imgur link above, is a detail of what blobs can run at each stage, and which blocks load which
<clever> green blobs are open source, red are closed
<mrvn> Do you happen to know what IP and services I need to highjack locally to load my own stuff?
<clever> yep, and good luck :P
<clever> let me find it again
mniip has quit [Ping timeout: 606 seconds]
<mrvn> Is it just dhcp, dns and hhtp for the 22MB boot.img?
<clever> https, with a private root CA that expires in 2121
<mrvn> clever: so it won't boot my own image then? Need to build new firmware with my own CA?
<clever> yep
<clever> but the same firmware ramdisk can be used over other channels
<mrvn> :( Maybe someone will make a tool to use the existing bootloader image and swap out the CA in it.
<mrvn> + url while you are at it
<clever> if you set boot_ramdisk=1 in config.txt
<clever> then the firmware will look for a boot.img, and load that instead
<clever> that works on anything the pi4 can already boot from
<clever> SD, usb, tftp, nvme
<mrvn> but then (tftp) it's unsecure. 8-P
<clever> you can also put an rsa2048 keypair of your choosing into the eeprom
<clever> and boot.img must then be signed with that keypair
<clever> any tampering with the boot.img will result in it not booting
<mrvn> that helps. Still not as nice as http(s)
<clever> having a public key in the eeprom will also force boot_ramdisk=1
<mrvn> I like the idea of being able to plug in a Pi anywhere in the world and it can download/install for a blank SD card.
<clever> also, the https requires you to hold shift before it downloads
<clever> so it wont run on bootup
<clever> yeah, thats very nice
<clever> mrvn: there are also some secure-boot bits that get in the way of changing the CA
<mrvn> I've been carrying around an USB stick for that for ages and now it's directly on a device.
<clever> the sha256 of bootmain, is baked into bootcode.bin
<clever> so after you patch bootmain, you have to patch bootcode.bin
<clever> and bootcode.bin is also signed with hmac-sha1 (and rsa, but that isnt checked by default)
<mrvn> that's ok for a tool to automate
<clever> and now you need the hmac-sha1 "secret" keys
mniip has joined #osdev
<mrvn> Is anythin in the maskrom?
<clever> the hmac key is an xor between a maskrom value and an OTP value
<clever> so it has support for per-device keys
dude12312414 has joined #osdev
<clever> but, that would mean a recovery.bin only works on one pi, and it would be a tech-support nightmare
<clever> so every single pi4, pi400, and cm4 uses the exact same hmac keys!
<clever> mrvn: this is the chunk of code you would have to patch in bootmain: https://paste.debian.net/1231332/
<bslsk05> ​paste.debian.net: debian Pastezone
<clever> you have a 16 bit size for the cert (an immediate in an opcode), and you have a pc-relative offset to the cert blob, all in a .elf file
mahmutov has quit [Ping timeout: 256 seconds]
<mrvn> st r1,(r0)? output parameter?
<clever> yeah, r0 is param_1, a pointer from the caller
<mrvn> and I assume 0x317 is the size of the cert?
<clever> yep
<clever> mrvn: and this is the full cert: https://paste.debian.net/1231333/
<bslsk05> ​paste.debian.net: debian Pastezone
<mrvn> So "all" I need is to make a CA that's <= 0x317, write it over the existing one and adjust the size then?
<clever> thats one option
<clever> another is to mess with the elf headers, to make the .data larger, or add a second LOAD
<clever> https://git.venev.name/hristo/rpi-eeprom-compress/ then you need to use this to compress the elf (and decompress before patching)
<bslsk05> ​git.venev.name: rpi-eeprom-compress - Unnamed repository; edit this file 'description' to name the repository.
<mrvn> Similar with the url, overwrite with a shorter one. Seems possible to customize without having to compile my own booloader. Only problem would be the hmac-sha1 you mentioned.
<bslsk05> ​gist.github.com: rpi eeprom decode · GitHub
<clever> and this details the format for files in the eeprom itself
<clever> you need to overwrite the bootmain file, which has a magic of 55aaf33f and a sha256 footer
<clever> and then you have to modify the bootcode.bin, magic of 0x55aaf00f
<clever> and bootcode.bin must be re-signed: https://github.com/librerpi/rpi-tools/tree/master/signing-tool
<bslsk05> ​github.com: rpi-tools/signing-tool at master · librerpi/rpi-tools · GitHub
<clever> mrvn: and to get the keys, compile LK into a start4.elf, boot it up, and then you can run https://github.com/librerpi/lk-overlay/tree/master/app/signing-dump
<bslsk05> ​github.com: lk-overlay/app/signing-dump at master · librerpi/lk-overlay · GitHub
<clever> it will spit out the key material in a form sign.js can consume
<clever> mrvn: from the ssl cert above, can you see any flaws with it?
<mrvn> 2121 is pretty optimistic
<mrvn> most of it is gibberish to me, so no.
<mrvn> It's a bit odd that it is self signed.
<clever> if you pinned the cert for a normal https server, that was signed by a real ca, it would expire fairly quickly
<clever> and then no rpi with old firmware can boot
<clever> how do you fix that?
<clever> and if you pin a real ca, you must always get your cert signed by them, and they have the ability to hand out fake certs when hacked or coerced
<clever> a private CA seems like a simple solution
<mrvn> Yeah. You don't have anything in the Pi to verify the signature anyway, so you don't have a known root to lend the CA trust.
<clever> yeah, its just whatever root you decide to bake into the 512kb eeprom
<clever> it also lacks an RTC, so you have no way to know if an expired cert is being dug out of a grave
<mrvn> Don't you normally have a super secret CA as root you keep in a safe, then a more exposed one you use and then a less capable cert for just the https connects?
<clever> oh, and your right
<clever> Issuer: C = GB, ST = England, O = Raspberry PI Limited, OU = Raspberry PI ECC CA, CN = Raspberry PI Intermediate CA, emailAddress = support@raspberrypi.com
<klange> entirely tangential, but you wouldn't happen to know why my rpi takes several minutes between the bootloader and the rainbow screen whenever I modify the boot partition, would you?
<clever> the cert on the site, reports `Raspberry PI Intermediate CA` as the issuer
<clever> but i dont have that cert
<klange> i should grab more ethernet cables and try to set up tftp boot...
<klange> and maybe also put my USB-C wall wart on something with a power switch...
<clever> klange: simplest option there, is to turn on the debug logs, and hook up a uart
<clever> if you run `sudo rpi-eeprom-config --edit` under raspi-os, you can change BOOT_UART to 1
<mrvn> klange: put a push button on the RUN pins.
<clever> and reboot it once
<clever> then it will print all bootloader logs to the uart
<klange> but presumably if it had anything to log it would be printing them to the graphical bootloader?
<clever> the hdmi doesnt turn on right away
<klange> like why even make such a big deal out of having added a graphical bootloader if it's not going to tell me anything helpful
<klange> it does
<clever> there is an HDMI_DELAY that stops it from turning hdmi on until it has spent X seconds trying to boot
<clever> because too many noobs assumed it was an error screen
<clever> so it has to hide from the noobs :P
<klange> but I am getting the screen
<mrvn> .oO(Did it have a blue background?)
<clever> what are you booting from?
<clever> sd? usb?
<clever> mrvn: black
<klange> sd
<clever> SD boot should be pretty quick
<mrvn> all noobs know error screens are blue
<clever> what about your boot order?
<klange> default
<clever> is noobs/pinn installed?
<klange> I'm reasonably certain it's after it determines it needs to boot from SD, but it's before ARM code starts
<clever> is it a delay before or after the rainbow?
<klange> Before
<clever> before or after the hdmi diagnostics screen?
<klange> after
<clever> thats an odd place for it to delay
<klange> It only happens immediately after I've written to the boot partition.
<klange> Subsequent boots are immediate
<clever> are you removing the card to write to it?
<klange> So I assume it's the VC doing an integrity check on the FAT or something
<klange> yes
<clever> ive never seen any signs of the VC having fsck support
<mrvn> are you umount, sync, eject?
<mrvn> do you ...
<klange> full process, yes, Linux host doing the modifications thinks the partition is fine
heat has quit [Remote host closed the connection]
<klange> The full process I am experiencing, if I apply power without the card to force the graphical diagnostic screen, is that I see that, it reports it is going to boot from SD after I insert the card, HDMI cuts off, several minutes pass (~3-4), I get signal back and see the rainbow screen, and then my OS loads.
<klange> If I power cycle after that point, it boots immediately.
<clever> that almost sounds like the EDID eeprom in your hdmi display is busted
<clever> and its not responding to the i2c commands
<klange> It happens on multiple displays, none of which have shown any signs of being borken.