<zid>
dogg and dre yea thinking about it it was definitely dogg singing
<zid>
any silly ideas for VBE?
YuutaW has quit [Quit: WeeChat 3.1]
immibis_ has joined #osdev
qookie has quit [Ping timeout: 255 seconds]
immibis has quit [Ping timeout: 246 seconds]
<kingoffrance>
i saw the gcc "global register variables" on mips supposedly lets you use coprocessor registers...so, i dont know, incorporate that into your psx port or patch gcc until you can
<kingoffrance>
and there are also "local register variables" but it says you can only pass to and fro asm()
<kingoffrance>
local as in a function
<kingoffrance>
that feature sounds more useful when you are using <registers that code will not use>
<kingoffrance>
i.e. coprocessor, other devices, etc.
<kingoffrance>
but i didnt see any instance except mips that was ever the case according to docs
<kingoffrance>
maybe it will be expanded someday
vdamewood has joined #osdev
<geist>
sure. that's actually useful on other arches too
<geist>
for example, on riscv the ABI calls out the 'gp' and 'tp' registers as being fixed. f you're the OS you can easily set them and/or access them directly via a global register
<geist>
or on arm64 you can set the x18 register (also free via the ABI) for your own purposes
<geist>
register current_thread *ct asm("x18"); for example
<kingoffrance>
i wrote a silly compile test to see what it allows, for ia32 it seems it can do 8-bit [a-d][hl] 16-bit [a-d]x 32-bit e[a-d]x, e[sd]i, e[bs]p those will compile anyhow. really, i think it likes ah or al but not both etc.
<moon-child>
ah is evil and should be destroyed
<kingoffrance>
it is, 8008 A, B-E . what is this 16-bit nonsense
<kingoffrance>
funny enough i can do register unsigned char foo[1] or 2 or 4 but 3 or 5 it gets made for e.g. eax. but you can access as "byte arrays" apparently (only did compile test of "declarations" -- no initializing allowed either)
<kingoffrance>
*gets mad foo[1] asm("eax")
<kingoffrance>
if you are doing this, you should know how big short/int/long etc. are
<kingoffrance>
so i dont know why you'd do that, but hey
<kingoffrance>
it doesnt want you pretending its 24-bit lol
<kingoffrance>
this might also be a problem if tehy started letting you do registers from multiple devices
<kingoffrance>
somehow has to "translate" to c types for your main cpu
<kingoffrance>
or, youd end up with e.g. "this int is type x, this other int is actually type y" etc.
dormito has joined #osdev
SGautam has joined #osdev
vdamewood has quit [Quit: Life beckons]
iorem has joined #osdev
<kingoffrance>
i wonder if would let you stick overlapping things in a union lol
<moon-child>
thanks, I hate it
gog has quit [Ping timeout: 250 seconds]
<kingoffrance>
its infinitely more useful if you have code to everything, then you can tell code not to touch things etc.
<kingoffrance>
*have source
<kingoffrance>
hey you said "what if custom callign convention" well...
<zid>
someone made a github account just to send me a Makefile s/rm/$(RM) patch
dude12312414 has joined #osdev
<kingoffrance>
the docs say dont need to make declaration included/visible everywhere, instead use ffixed-<reg> whatever to warn other code, which i think is another way of saying "relies on the honor system"
<kingoffrance>
i.e. sounds like the linker will not stop you from doing something conflicting
<geist>
kingoffrance: right
<geist>
or writing asm, etc. the linker dont know what the code does
<kingoffrance>
well thats not bad necessarily, gets out of your way. you just wont know until you crash at runtime if you messed up :)
<kingoffrance>
it means it is your responsiblity to make sense
<kingoffrance>
its the ghost of unix past lol
<doug16k>
kingoffrance, on x86_64, you can use the "l" on all 8 registers. if the instruction has a rex prefix, you can't use any "h" register. It would have a rex prefix if it has 64 bit operand size or uses any of the added registers r8-r15 and [xyz]mm8-[xyz]mm15
<doug16k>
all 16
<doug16k>
r15l
<doug16k>
so compilers don't have to screw around with 8 bit values not working in esi edi ebp esp
<doug16k>
skylake has errata with h registers. you should stop using them altogether
<kingoffrance>
youll be 64 by the time i get to 64-bit
<doug16k>
they can't fix it in microcode
<doug16k>
lol
<doug16k>
actually I think they may have a fix
<doug16k>
I'm curious how much it cripples it
<doug16k>
haskell was hammering h registers and they had failures with their codegen
<doug16k>
you might as well consider the h registers gone in long mode, since rex will annoyingly cause issues with being able to use them
freakazoid333 has quit [Read error: Connection reset by peer]
sts-q has joined #osdev
<doug16k>
you want your code to never pick at registers. ideally you create completely new full width results every time
<doug16k>
movzbl is probably going to run better than mov al,
<doug16k>
because picking at registers makes a sequential section. if you create a whole new register, it breaks dependency of destination and starts a brand new dependency chain
<doug16k>
it could be able to start it far ahead of time
<doug16k>
point being, byte registers are bad
<doug16k>
for store of low 8 bits, it's fine - internally you are depending on that whole register
isaacwoods has quit [Quit: WeeChat 3.2]
riposte has quit [Ping timeout: 256 seconds]
<kingoffrance>
makes sense. well i try not to hardcode anything, so im not sure i want to use gcc feature
<kingoffrance>
i just wondered waht registers it would allow
freakazoid333 has joined #osdev
<doug16k>
realistically you would use a callee saved register, so you can link with code that is oblivious about it, and it still works
<doug16k>
ebx esi edi would be your options
<doug16k>
bl bh I guess
riposte has joined #osdev
<doug16k>
if you used fomit-frame-pointer, you could use ebp
<moon-child>
doug16k: you don't need a rex prefix for *mm8-15, if you use the vex prefix instead
ElectronApps has joined #osdev
nshp has joined #osdev
nyah has quit [Ping timeout: 255 seconds]
dude12312414 has quit [Quit: THE RAM IS TOO DAMN HIGH]
YuutaW has joined #osdev
vdamewood has joined #osdev
jjuran has quit [Quit: Killing Colloquy first, before it kills me…]
jjuran has joined #osdev
ElectronApps has quit [Remote host closed the connection]
rorx has quit [Ping timeout: 245 seconds]
doug16k has quit [Remote host closed the connection]
doug16k has joined #osdev
doug16k has quit [Remote host closed the connection]
doug16k has joined #osdev
SGautam has quit [Ping timeout: 258 seconds]
SGautam has joined #osdev
mctpyt has joined #osdev
MarchHare has joined #osdev
MarchHare has quit [Ping timeout: 240 seconds]
SGautam has quit [Ping timeout: 255 seconds]
SGautam has joined #osdev
mcfrdy has quit []
immibis_ has quit [Ping timeout: 258 seconds]
GeDaMo has joined #osdev
Burgundy has joined #osdev
immibis__ has joined #osdev
immibis__ is now known as immibis
mcfrdy has joined #osdev
<klange>
ay, we got kernel modules again
<sham1>
Nice
<sham1>
Those are good to hvae
<zid>
can you [*] and [M] them or just [M]
<klange>
only supports runtime loading, not compile time linking, mostly because I did the minimum to have a clean implementation of the object linker and get a couple of drivers moved out of the core kernel
<klange>
going to leave a lot more in the main kernel image this time around; toaru32 had critical features in modules so you had to load them at boot time; sticking with stuff I can load from the ramdisk during startup now
<bslsk05>
github.com: toaruos/04_modprobe.sh at master · klange/toaruos · GitHub
<klange>
Doesn't even support modules depending on other modules yet; the old one sort of did - at least it could bind symbols from other loaded modules, so I'll add that eventually...
<klange>
I have plans... like sticking PCI IDs into a special section so they can be automagically checked by a userspace device manager rather than that hand-written shell script.
<zid>
I'd like to find out why perf is so bad, but not enough to actually do anything
ElectronApps has joined #osdev
gw has joined #osdev
<zid>
It runs about realtime speed, but without any syncing
<zid>
it's just an infinite loop that writes framebuffer pixels
<doug16k>
I got over 1000 fps in kvm
<zid>
yea it gets 1200fps on my desktop
<doug16k>
full memory bandwidth basically
<doug16k>
need to profile it
<zid>
It might just be crappy fb perf in general and the emu just gets that long between qemu code
<doug16k>
can add your own hardware perf counter profiler with a perf counter set on cycles. set counter to -100000 and configure perf INT in LAPIC to get interrupt every 100000 cycles
<doug16k>
return address is sample
<doug16k>
reset perf counter to -100000 or whatever before returning
<zid>
I think the perf loss is imaginary though
<doug16k>
then blast them to serial? script to lookup function from nm dump?
<zid>
and I don't have lapic :P
<doug16k>
you have timer irq?
<doug16k>
just collect the return addresses
<zid>
okay but that will just show 100% cpu use inside my code, is the point
<zid>
I think the cpu loss is inside qemu's memcpy to its texture buffer or whatever
<doug16k>
yes but you remember them in a giant array and write them to output, then analyze all the saved PC against nm symbol addresses
<doug16k>
then you can calculate percent of samples in each function
<doug16k>
then sort by that descending
<zid>
and they will add to 100%
<doug16k>
then fix perf of top one
<zid>
and tell me nothing
<zid>
I know the perf of the *program*
<zid>
The question is why is the perf of the qemu-program system bad
<doug16k>
...then analyze all the saved PC against nm symbol addresses...
<zid>
and they will add to 100%
<zid>
and tell me nothing
<doug16k>
it will tell you which *function* it was in the most
<zid>
you're not listening
<zid>
I now which function it's in the most
<doug16k>
how do you know?
<zid>
because it's the same program as I run on my desktop
<doug16k>
what if it is taking way more cycles in something else?
<zid>
and I wrote all the code
<zid>
what something else
<doug16k>
what if some port I/O is 1000x slower
<zid>
It doesn't use any i/o ports
<zid>
and it's in userspace.
sortie has quit [Quit: Leaving]
<doug16k>
why is it in qemu?
<zid>
because my OS is hosting it.
sortie has joined #osdev
<doug16k>
what I said applied to the OS
pg12 has quit [Ping timeout: 268 seconds]
<zid>
good news, the OS isn't running
<zid>
>it's just an infinite loop that writes framebuffer pixels
<doug16k>
so the zelda emulation was just to mislead
<doug16k>
make me think it's an emulator
<zid>
You're trolling yourself and blaming me
<zid>
It is an emulator.
<zid>
The emulator is an infinite loop, that writes pixels to a framebuffer.
<zid>
It uses 100% cpu inside the guest, NO MATTER WHAT
pg12 has joined #osdev
<zid>
If you profile it, you will get precisely: 100% cpu use in 'the code' and the split will be 'what zelda does'
<zid>
it's completely useless to measure it, when we're asking about *qemu* performance
<klange>
vmport mouse, which is why it's loaded in qemu
<zid>
ahh, is it tied into the vmware balloon stuff, or just the name you gave to vmware's mouse?
<klange>
also there's some commented out code for doing automatic modesetting in vmware itself that I should probably uncomment
<zid>
I know vmware has a bunch of backchannel stuff
<klange>
I don't know how the balloon stuff works but presumably it's the same interface
<klange>
oh i remember why that was commented out, it was waiting on kernel tasklets
<klange>
those have existed for ages, but I wanted the absolute mouse support in qemu really early on
<zid>
MIce? Where we're going we don't need mice!
<zid>
I don't even have keyboard support
<zid>
I do have utf-8 text rendering in my dmesg though, in case some wants a lovely monospaced katakana
<zid>
My plan was always to interface with the kernel over http though
<zid>
so like a modem or whatever, where you web configure it and then it just does stuff in the bg
<klange>
I'm spoiled.
<klange>
I got really annoyed about having to un-grab the mouse in qemu during early testing once I got my GUI up on the new kernel, so I _had_ to get the vmport mouse implemented or I was gonna go apeshit.
<zid>
oh is that how that works
<zid>
it lets you make it act like a window?
<zid>
I'm using qemu in vmware so I'm double-nested in mouse shenanigans
<klange>
qemu also supports a usb tablet, but that obviously needs a usb stack
<klange>
the vmport mouse also works with the vnc frontend, which is really important
<zid>
my shit does boot on VMware, but I used the bochs-vbe device for my test geraffiks
<zid>
what doc do I need for vmware graphics?
<klange>
search 'vmware svga' and you might find info
<klange>
it's roughly the same process as the bochs vbe, but different ports
<zid>
not that I touch ports
<zid>
I do it over the pci-e mmio region
<zid>
it emulates it as ports though
<klange>
Careful with that, virtualbox doesn't support the mmio mode.
doug16k has quit [Quit: Leaving]
<zid>
vbox hadn't even entered my mind
<zid>
not sure I've ever used it
<klange>
Welcome to osdev, test your shit in everything :)
<zid>
My real hw is an e1000 I'm tempted to try booting it and pinging it
<zid>
the problem is, I'd have nothing to ping it /with/
<klange>
Don't forget to have a ten year old laptop constantly doing nothing in a corner of the room...
<klange>
64 bytes from 192.168.11.42: icmp_seq=1 ttl=64 time=0.940 ms
<klange>
:3
<zid>
try 192.168.222.0/24, you may need to bring your own ethernet cable
mhall has quit [Quit: Connection closed for inactivity]
<klange>
oh right, if I load my i965 modesetting driver during init, then it'll change the resolution after the splash screen has initialized...
<zid>
[*] time
<klange>
and the splashscreen doesn't handle display size changes...
<zid>
I thought briefly about how I'd support [*] and decided "it's hard"
<zid>
I guess like.. x-macros to make an array of init functions to call
<klange>
the compositor does - there's an interface to send a signal to the display "owner" when the resolution changes, just the splash screen doesn't do anything with it :)
<zid>
which should make the linker happy about not tossing it away
<klange>
so I get garbled text for a bit before the compositor comes up, lol
gw has quit [Quit: Konversation terminated!]
dennis95 has quit [Remote host closed the connection]
dennis95 has joined #osdev
qookie has joined #osdev
ZetItUp has joined #osdev
yomon has joined #osdev
Burgundy has quit [Ping timeout: 255 seconds]
yomon has quit [Ping timeout: 252 seconds]
immibis_ has joined #osdev
immibis has quit [Ping timeout: 255 seconds]
gog has joined #osdev
nyah has joined #osdev
Oli has joined #osdev
Oli has quit [Client Quit]
tacco has joined #osdev
sortie has quit [Remote host closed the connection]
sortie has joined #osdev
brynet has quit [Quit: leaving]
MarchHare has joined #osdev
brynet has joined #osdev
immibis_ is now known as immibis
myon98 has joined #osdev
Skyz has joined #osdev
andreas303 has quit [Ping timeout: 268 seconds]
isaacwoods has joined #osdev
andreas303 has joined #osdev
opios2 has quit [Ping timeout: 268 seconds]
rorx has quit [Ping timeout: 268 seconds]
dh` has quit [Ping timeout: 268 seconds]
meisaka has quit [Ping timeout: 268 seconds]
meisaka has joined #osdev
rorx has joined #osdev
nismbu has quit [Ping timeout: 256 seconds]
Mutabah has quit [Ping timeout: 268 seconds]
Mutabah has joined #osdev
ElectronApps has quit [Read error: Connection reset by peer]
froggey has quit [Ping timeout: 256 seconds]
froggey has joined #osdev
vdamewood has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
qookie has quit [Quit: leaving]
iorem has quit [Quit: Connection closed]
Robbe has joined #osdev
zoey has joined #osdev
sortie has quit [Remote host closed the connection]
sortie has joined #osdev
<Skyz>
Klange: Do you want people to use your OS?
<Skyz>
I would suggest making an iso that someone could try loading into virtual box
<dzwdz1>
are there any executable formats simpler than a.out? ones that don't require any relocation
jjuran has joined #osdev
<clever>
dzwdz1: do raw .bin files count?
<dzwdz1>
but that would require the executable to be built as a PIE, right?
<clever>
you could just set a rule that its always loaded to a given addr
<zid>
Yea the opposite of relocatable just non-relocatable :P
<clever>
as an example, the bootcode.bin file on an rpi, just gets copied to 0x8000_0000 physical, and then the rom jumps to 0x8000_0200
<clever>
no relocation, no patching
<zid>
tbh though, a non-relocatable ELF is a pretty damn simple format
<GeDaMo>
Does source code which is compiled at load time count as an executable format? :P
<zid>
you can just read a certain offset pair and you're done
<dzwdz1>
wouldn't i need to load everything into rw memory if i was using a .bin file?
<clever>
dzwdz1: yeah, since there is no way to define what permissions go where
<dzwdz1>
how do microkernels that don't have an exec() usually handle loading the first executable?
<dzwdz1>
do they have an elf parser just for it?
<zid>
My OS jsut has two images
<zid>
one loads to -2GB ones loads to 2MB
<zid>
and the kernel jumps to 2MB
<dzwdz1>
my idea is that the kernel loads an executable which has the initrd drivers
<dzwdz1>
and which also has the initrd embedded in itself
<GeDaMo>
Why can't the executable just be loaded to RO memory?
<dzwdz1>
and the elf loader, since i don't see the point in putting it in the kennel
<dzwdz1>
.data?
<clever>
GeDaMo: stack
<dzwdz1>
unless there's a way to prevent the compiler from using .data at all
<clever>
it needs some kind of rw memory, to be able to make a few func calls to fixup its own permissions
<GeDaMo>
A default stack space doesn't seem like a big problem
<clever>
yeah, that could work around it
<dzwdz1>
^
<dzwdz1>
it's only .data which is the issue
dzwdz1 is now known as dzwdz
<clever>
and you have to define the size of .bss somewhere
<GeDaMo>
The executable requests RW memory and copies across the data initial values
<GeDaMo>
Or changes the permissions on part of itself which has the data values
<clever>
a lot of MCU's go the copy route
<clever>
the .text and a template of .data are all in rom (or flash that isnt modified)
<clever>
and on startup, it copies .data to ram
<GeDaMo>
Yeah, that's what I was thinking of
<dzwdz>
does gcc support that?
<clever>
dzwdz: yep, its handled by the linker script
<zid>
As in, it's nothing to do with gcc
<clever>
you can setup the linker script so the virtual addr (used by compiled code) is in ram, but the physical address (only used by the loader/programmer) is in flash/rom
<clever>
then whatever is generating the .bin file for your rom, has to respect physical addresses in the .elf
<clever>
but the linker uses virtual when filling things in on the asm gcc provided
<dzwdz>
that seems more complicated than just rolling my own executable format tbh
<clever>
dzwdz: you would rather that every user has to have a custom binutils?
<zid>
I mean, that is how you'd make your own executable format
<clever>
the above process, lets end-users program the MCU with stock tools
<dzwdz>
clever: it would only be used for a single executable, which would be the same in most setups
<clever>
dzwdz: sounds like an MCU, it runs a single binary on startup, and it never changes
<zid>
. = VIRT; .text { * (.*); } is a pretty normal linker script for something like that
<zid>
which just makes all the code expect to be loaded to `VIRT`
<zid>
and all the data and stuff be included int he output
<dzwdz>
doesn't the compiler expect a writeable .data section?
<zid>
If you want permissions on various pieces of memory, you're going to need a proper image format
<bslsk05>
github.com: pico-sdk/crt0.S at master · raspberrypi/pico-sdk · GitHub
<clever>
229 gets the address of a set of things to copy
<clever>
236 copies one thing, and then it repeats
<zid>
ELF is just a main header that tells you where to find the program headers, and how many of them there are
<zid>
and each program header tells you where and how many bytes to copy
<clever>
278-295 is a set of source/start/end pointers
<dzwdz>
it just seemed like a complicated format
<zid>
everything else is for linkers and debuggers and stuff
<clever>
the linker script says to put the .data into flash after .text
<clever>
so the .S file i linked, can safely assume that .data begins at __etext
<clever>
and because .text stays in flash, __etext is a flash based addr
<clever>
which jsut happens to be where .data begins in flash
<zid>
try readelf -l on some binary
<zid>
It's a 6 field array
<zid>
offset/virt/phys/fileoffset/size/flags
Skyz has quit [Quit: Client closed]
ZipCPU has quit [Ping timeout: 272 seconds]
ZipCPU has joined #osdev
* geist
yawns
<geist>
good afternoon folks
<nur>
afternoon sir
<dzwdz>
hi o/
<nur>
although it is 3am here
lucf117 has joined #osdev
<dzwdz>
why are you still up? that's not healthy
<geist>
huh 3am what time zone is that?
<geist>
i thought you were in europe
rorx has quit [Ping timeout: 240 seconds]
<geist>
east asia i guess, or west asia
<nur>
Malaysia
<nur>
I'm up because well, the rest of my people live opposite the world and this is when _they_ are up
<nur>
why did you think I was in Europe
rorx has joined #osdev
<geist>
i dunno, i just thought for some reason you had mentioned you were in scandinavia before
<geist>
so many people i've talked to here, easy to get them mixed up
<geist>
also maybe i have a bias with words like 'nur' sounding like 'north' and thus scandinavia comes into mind
mcfrdy has quit [Ping timeout: 255 seconds]
<Oli>
exit
<Oli>
whoops
mcfrdy has joined #osdev
Oli has quit [Quit: leaving]
Oli has joined #osdev
Oli has quit [Quit: leaving]
Oli has joined #osdev
<sortie>
Tonight I'm poking a bit on my build infrastructure. A bunch of the release logic, like generating the html manual, is actually not part of the ‘make release’ command but are done by an unversioned CI script
<sortie>
So I'm basically merging stuff from that CI script into the main repository, plus making sure it works natively. Turns out grep -r --include='*.html' crashes with a libc assertion failure in mbsnrtowcs(3)
<sortie>
One thing I never tried is cross-compiling my OS under itself, i.e. building the 32-bit release under the 64-bit release, with a proper cross-compiler. It should work in theory, but some ports might have stupid kinds of issues, who knows. Certainly it'd simplify things for me if I could do this.
<sortie>
Anyways I'm starting to finally get over this nasty cold and feel somewhat productive again and I have coffee :)
<sortie>
What are people up to tonight?
<gog>
gonna smoke a bowl and watch a thing about black holes
srjek has joined #osdev
<sortie>
Fun :)
jjuran has quit [Ping timeout: 250 seconds]
jjuran has joined #osdev
isaacwoods has quit [Read error: Connection reset by peer]
GeDaMo has quit [Quit: Leaving.]
vai has joined #osdev
<vai>
howdy all
<vai>
sortie:
<sortie>
Hiya vai
<sortie>
How are you?
mcfrdy has quit [Ping timeout: 255 seconds]
mcfrdy has joined #osdev
silverwhitefish has quit [Ping timeout: 258 seconds]
heat has joined #osdev
<heat>
warm greetings fellow humans
freakazoid333 has quit [Read error: Connection reset by peer]
<clever>
heat: then i have you to thank for this weather? :D
<heat>
:D
<heat>
today peaked at around 30C around here
<heat>
not too bad, but not great
<mjg>
how hot is it in russia?
<heat>
29 in saint petersburg
<mjg>
if it's anything above 15C they are getting cooked alive
<heat>
25 and raining in vladivostok
<zid>
>20C still outside at midnight, bleh
<zid>
it's probably like 30C in here
dude12312414 has quit [Quit: THE RAM IS TOO DAMN HIGH]
<heat>
so I had this idea, what if I wrote a new build system for building other projects that have separate build systems
dude12312414 has joined #osdev
<heat>
it would work really well for an OS monorepo that has lots of different random projects/build systems/packages
<heat>
has anyone tried something similar?
<zid>
Yes, I sneak into the houses of people who use ant/ninja/cmake/etc and stab them to death.
<zid>
I find it more effective ovrall
<heat>
what happens to kati users?
<heat>
stab or no stab?
<zid>
no idea what that is, I can't be everywhere at once sadly
<heat>
android thing that takes a makefile and outputs a ninja file
<zid>
android people are already insane, so them doing insane things is to be expected
<heat>
ah I see
<heat>
you must be one of those autoconf users
<kazinsal>
autotools is heretical
<clever>
most of my stuff builds with just a makefile, no autotools/autoconf
<clever>
only if i'm targetings multiples arches, do i get cmake involved
<clever>
like windows/linux/mac
<heat>
yeah but the issue here is: what do you do when you have a monorepo?
<clever>
my most complex monorepo is rpi-open-firmware
<heat>
regular build systems don't understand when other build systems need to rebuild/have rebuilt
<clever>
it builds for linux-arm, baremetal-arm, baremetal-vpu
<clever>
and that is all done with plain makefiles (and the nix package manager)
<clever>
nix handles the rebuilding for you
<clever>
basically, every input dependency is at a path like /nix/store/hash-name
<zid>
Packaging is for distros
<clever>
and you must install to /nix/store/hash-name, which is a hash of all of your input strings
<zid>
we honestly don't need to be detecting if various packages are installed in ./configure
<zid>
gentoo shouldn't let you emerge the ebuild until it has the deps, or the ebuild is broken
<heat>
zid, you do if they're optional
<clever>
yeah, my makefile just assumes that -l and #include can find things
<clever>
its up to the package manager to make it discoverable and installed
<zid>
heat: You still have to specify that, why bother specifying it to configure
<clever>
zid: nix is basically gentoo, but better
<zid>
like, I'm surprised it isn't make -DOPTION instead of ./configure --with-OPTION
<zid>
or such
<clever>
as an example, if you build something with gentoo, it can find things you didnt enable with a USE flag
<clever>
and then link against them
<clever>
but because you didnt set the USE flag, gentoo thinks its not a dependency
<zid>
USE should set ./configure --with
<clever>
then things break when you uninstall it
<zid>
if not, ebuild bug
<clever>
nix cant have that bug
<zid>
good for nix
<clever>
so its better then gentoo
<clever>
/usr/lib doesnt exist, /lib doesnt exist
<zid>
gentoo is shit because python
<clever>
the only way to find a package, is for the package manager to tell you where it is
<zid>
not because of how portage works
<clever>
if you dont ask for something, youll never know its installed
<heat>
anyway, my idea here was to create some sort of meta-build-system language that describes the build and do the rest in Go/Rust (to force myself to learn one of those)
<clever>
heat: sounds like nix :P
<heat>
that would fix the Make/ninja/gn/cmake/bazel stupid
<clever>
nix describes the dep tree, what binary to run to kick-off a build, and what env vars to set during the build, and thats it
<zid>
heat: you just invented ebuilds, except awful awful python
<clever>
built ontop of that, is a framework to automate ./configure && make && make install, but other frameworks exist for other languages
<heat>
clever: does it know how to check if something was rebuilt?
<clever>
heat: basically, the product gets installed to /nix/store/$(hash depa-1.0 depb-2.0)-result
<bslsk05>
github.com: gentoo/gcc-4.7.3-r1.ebuild at master · ClusterHQ/gentoo · GitHub
<zid>
random ebuild google gave me
<clever>
heat: if the version of depa changes, the hash changes, so the build product isnt out of date, its just missing
<clever>
heat: every time you ask for a given package, it computes that hash, and then checks if you have a copy locally, matching that exact build plan
<zid>
It's just a bash script that sets up configure and make
<zid>
based on your system settings
<clever>
if ANY part of the build plan differs, its considered a different result, and rebuilds what it needs to
<heat>
1) crazy duplication? 2) yeah but what happens for changes in the actual build
dennis95 has quit [Quit: Leaving]
<clever>
heat: the hash covers the build directions, and the hash for every input library
<heat>
package foo depends on A and B, A and B didn't change but foo's source has changed
<clever>
changing the source impacts the hash
<heat>
they also hash the source?
<clever>
yes
<heat>
that's erm... possibly not practical
<clever>
gentoo does it too
<clever>
the hash of the .tar.gz is in the package description
<clever>
in nix, the hash for the entire build plan, is rooted on those .tar.gz hashes, and the build instructions defined in the nix files
<clever>
and at download time, if a given .tar.gz doesnt have the expected hash, its a fatal error, and the build aborts
<zid>
just ship binaries lol
<heat>
yes but I'm not dealing with tarballs and if something that isn't built changes, I don't care
<clever>
zid: because of how reproducable nix is, you can now share binaries between machines
<heat>
and if the build config changes, I *do* care
<clever>
heat: something local?
<zid>
great, now get rid of all that mess
<zid>
and just provide binaries
<heat>
clever: hm?
<clever>
heat: if your not dealing with a tarball, how are you getting the source?
<bslsk05>
github.com: rpi-open-firmware/default.nix at master · librerpi/rpi-open-firmware · GitHub
<clever>
heat: this tells nix to make a copy of ./. but ignore .o files as its copying
<clever>
it will then hash that copy
<clever>
and if that hash changes, rebuild
<clever>
zid: nix allows you to reuse the pre-built binaries from a cache, but then override anything you want, and it will seamlessly rebuild what is needed
<clever>
zid: so i can trivially add a patch to udev to fix a bug, and not need to worry about what i have to rebuild afterwards
<clever>
and still benefit from the pre-built binaries in the cache
<zid>
neat, neither do I
<heat>
thinking about this is hard
<clever>
heat: basically, the build instructions for a single package get compiled down to 3 things
<clever>
1: a binary to run to start the build (usually bash)
<clever>
2: the args for that binary (usually a bash script)
<clever>
3: a set of env vars to be set when launching that
<clever>
that whole thing gets serialized and hashed, to compute the output dir, /nix/store/${HASH}-${NAME}
<clever>
if you depend on another library, the ${HASH} of that libraries build plan, will be in one of the env vars
<clever>
so changing that library in any way, changes the env vars, and changes your own HASH
<clever>
to get rid of insanity like rebuilding when the version of curl changes, the package to download a URL for a .tar.gz has slightly different rules
<clever>
the HASH is based on the expected hash of the .tar.gz, not the binaries ran to fetch it
freakazoid333 has joined #osdev
<heat>
what I was thinking of was something like: you provide build configuration, platform/target/board/configuration, etc; build gets divided into stages (so boot stage (for things that go on the boot partition), system stage (for general system stuff)); you can't have dependencies between stages and a package that is used in multiple stages needs to be build N times
<clever>
nix is split up, and can do stages like that, but it can also reuse when it makes sense
<heat>
the build tool (with a cool name of course) would invoke the build of every package required by reading the metadata and having a general query command and build command(s)
<bslsk05>
github.com: rpi-open-firmware/default.nix at master · librerpi/rpi-open-firmware · GitHub
<clever>
heat: as an example, common on line 37 gets built in 3 different configurations, always as a static library
<clever>
baremetal arm, baremetal vpu, and linux arm
<clever>
tlsr (a heap library) on 27 gets built for just baremetal arm
<clever>
tlsf*
<clever>
chainloader on 72 gets build for baremetal-arm
<heat>
build output would get sent to something like out/$target-$board-$platform/$stage_name/path_to_pkg/$pkgname; and they would populate some sort of sysroot at something like out/$blahblah/$stage_name/sysroot
<clever>
firmware on 95 gets built for baremetal-vpu, and depends on the bare-arm build of chainloader
<clever>
bootdir on 175, depends on firmware
<clever>
and also compiles the dts files
<clever>
so if i ask for the bootdir, nix will built tlsf, common, chainloader, and firmware, in roughly that order
<heat>
in the end the build tool would take care of any stripping that may be required, and would assemble the partitions; it could also be needed to discard some files that were only needed by the build(like libc.a in some libc-devel kind of thing)
<clever>
diskImage on 196, will then make a .img file, containing an entire OS install, ready to boot
<clever>
with nix, libc.a is in the -dev outputs
<clever>
and nix searches your build product for paths, to see if you refer to something
<clever>
if you dont refer to -dev, then -dev doesnt get included in the disk image
silverwhitefish has joined #osdev
<heat>
right
<heat>
but shouldn't -dev packages be required to be explicitly installed by whoever is configuring the build/doing sysadmin?
<clever>
heat: when you define glibc as a dependency, the static folder gets its /lib added to the -L path, so gcc can find it with you `-lc`
<heat>
oh yeah something completely random: I found a decent reason to actually have a usr/ and have the root lib,bin symlink to it: the inode/block allocation is different in non-root so everything in usr/ will be closer together
<clever>
and the -dev folder gets its /include added to -I, so #include just works
<clever>
AFTER your build is done, nix will search your resulting binary, for both 5win19g9qga6fgf5wiwc6qhlfx3m2768 and f8ahd6cxx9ia4fzdiyczmhcv3c2qmd42
<clever>
and only if you have one of those hashes in your binary, do you depend on it
<clever>
if you dont delete on it, then it can be deleted or not copied
<heat>
like a pure binary grep?
<clever>
yeah
<heat>
that seems a bit less useful
<clever>
if 5win19g9qga6fgf5wiwc6qhlfx3m2768 doesnt appear in your result, there is ZERO way for you to ever find that libc.a
<heat>
why do those different outputs have different hashes?
<clever>
so you cant use it at runtime
<clever>
the hash also includes the output name, static/dev
sav has joined #osdev
dude12312414 has quit [Quit: THE RAM IS TOO DAMN HIGH]
<clever>
and if you cant use it at runtime, you wont care/notice if the path is missing at runtime
<heat>
interesting
<clever>
so nix is free to delete the entire -static during garbage collection
<clever>
or just omit it from the disk image
mcfrdy has quit [Ping timeout: 268 seconds]
<heat>
nix is certainly a lot closer to something I would want as a build-my-os tool
<clever>
nix also has native support for cross-compiling
<clever>
so you can just define how to build a package for any arch, then ask for 3 copies of it, for 3 different arches
<bslsk05>
github.com: nixpkgs/make-derivation.nix at master · NixOS/nixpkgs · GitHub
<clever>
heat: and there is the source behind it
<clever>
lines 257-259 convert the configureFlags from the package into a uniform style (an array of strings)
<clever>
260-262 then append elements to the list, if certain conditions are met
radens has joined #osdev
<clever>
configurePhase in setup.sh, then runs ./configure, with $configureFlags as flags
<radens>
Hey, is there a good datasheet to cite in my code for the programming interface for the serial/COM/UART ports for x86 PCs?
<radens>
Also, what's a good standard name for this interface? The OS Dev wiki has similar information on pages labelled COM, Serial, and UART.
<clever>
heat: its also extensible, if you add cmake to your runtime deps, it will (by default) hijack configurePhase, and modify it to do `mkdir build ; cd build ; cmake ..` instead
<heat>
is there any requirement for out-of-tree builds?
<heat>
or can packages do as they please?
<clever>
heat: for most packages, its just a dumb `./configure && make && make install`, but cmake by design allows for more out-of-tree stuff, so it defaults to using a build dir
<clever>
the temp dir it builds in, gets deleted automatically once the build is done
<heat>
also are there implicit dependencies on glibc/gcc libs/llvm libs/rust stuff?
<heat>
or are you required to specify that
<clever>
heat: gcc and glibc are in the default scope, but there are variants of the default env