klange changed the topic of #osdev to: Operating System Development || Don't ask to ask---just ask! || For 3+ LoC, use a pastebin (for example https://gist.github.com/) || Stats + Old logs: http://osdev-logs.qzx.com New Logs: https://libera.irclog.whitequark.org/osdev || Visit https://wiki.osdev.org and https://forum.osdev.org || Books: https://wiki.osdev.org/Books
SpikeHeron has joined #osdev
<geist> yeah that's fair
justmatt has quit [Remote host closed the connection]
<noeontheend> Tangential, but that's something I really appreciate about Nora Sandler's compiler tutorials. They contain absolutely no code and instead just give you broad outlines
dude12312414 has quit [Ping timeout: 268 seconds]
<bslsk05> ​norasandler.com: Writing a C Compiler, Part 1
<noeontheend> Something like that would be great for osdev
knusbaum has joined #osdev
<geist> Yah not bad
chartreuse has quit [Read error: Connection reset by peer]
knusbaum has quit [Quit: ZNC 1.8.2 - https://znc.in]
chartreuse has joined #osdev
knusbaum has joined #osdev
chartreuse has quit [Ping timeout: 272 seconds]
foudfou has quit [Ping timeout: 268 seconds]
foudfou has joined #osdev
<PapaFrog> What do you think of Crafting Interpreters?
<klys> i make a nybble code interpreter last month, thinking i could get it to turing complete phase, just I haven't had the time. it was almost there.
Vercas1 has joined #osdev
Vercas has quit [Ping timeout: 268 seconds]
Vercas1 is now known as Vercas
* PapaFrog hands klys a long piece of paper and a pencil.
<klys> heaven knows I need to pay attention
jack_rabbit has joined #osdev
knusbaum has quit [Ping timeout: 268 seconds]
Vercas has quit [Quit: Ping timeout (120 seconds)]
Vercas has joined #osdev
Vercas has quit [Remote host closed the connection]
foudfou has quit [Remote host closed the connection]
foudfou has joined #osdev
Vercas has joined #osdev
psykose has quit [Remote host closed the connection]
dude12312414 has joined #osdev
psykose has joined #osdev
[itchyjunk] has quit [Ping timeout: 268 seconds]
vdamewood has joined #osdev
gxt has quit [Write error: Broken pipe]
Vercas has quit [Write error: Broken pipe]
dude12312414 has quit [Remote host closed the connection]
foudfou has quit [Remote host closed the connection]
foudfou has joined #osdev
Vercas has joined #osdev
foudfou has quit [Remote host closed the connection]
foudfou has joined #osdev
foudfou has quit [Client Quit]
foudfou has joined #osdev
dude12312414 has joined #osdev
troseman has quit [Quit: Textual IRC Client: www.textualapp.com]
qubasa has joined #osdev
xhe has quit [Ping timeout: 260 seconds]
xhe has joined #osdev
dude12312414 has quit [Quit: THE RAM IS TOO DAMN HIGH]
xhe has quit [Ping timeout: 244 seconds]
xhe has joined #osdev
gog` has quit [Ping timeout: 272 seconds]
Vercas has quit [Remote host closed the connection]
zaquest has quit [Remote host closed the connection]
Vercas has joined #osdev
zaquest has joined #osdev
xhe has quit [Ping timeout: 244 seconds]
xhe has joined #osdev
chartreuse has joined #osdev
pretty_dumm_guy has quit [Quit: WeeChat 3.5]
foudfou has quit [Ping timeout: 268 seconds]
xhe has quit [Ping timeout: 240 seconds]
xhe has joined #osdev
tsraoien has joined #osdev
heat has quit [Ping timeout: 264 seconds]
tsraoien has quit [Ping timeout: 268 seconds]
bradd has joined #osdev
foudfou has joined #osdev
<klange> PapaFrog: Crafting Interpreters was downright life-changing
xhe has quit [Ping timeout: 240 seconds]
xhe has joined #osdev
Vercas has quit [Remote host closed the connection]
Vercas has joined #osdev
ajr has quit [Quit: WeeChat 3.6]
Vercas has quit [Remote host closed the connection]
Vercas has joined #osdev
foudfou has quit [Remote host closed the connection]
foudfou has joined #osdev
xhe has quit [Ping timeout: 272 seconds]
xhe has joined #osdev
bradd has quit [Quit: No Ping reply in 180 seconds.]
bradd has joined #osdev
xhe has quit [Remote host closed the connection]
Vercas has quit [Remote host closed the connection]
Vercas has joined #osdev
MiningMarsh has quit [Quit: ZNC 1.8.2 - https://znc.in]
MiningMarsh has joined #osdev
ThinkT510 has quit [Quit: WeeChat 3.6]
ThinkT510 has joined #osdev
sikkiladho_ has joined #osdev
h4zel has joined #osdev
xhe has joined #osdev
xhe has quit [Remote host closed the connection]
h4zel has quit [Ping timeout: 244 seconds]
Vercas has quit [Quit: Ping timeout (120 seconds)]
Vercas has joined #osdev
h4zel has joined #osdev
the_lanetly_052_ has joined #osdev
elastic_dog has quit [Ping timeout: 272 seconds]
elastic_dog has joined #osdev
opal has joined #osdev
<mrvn> What's that document called that explains the rules for checking memory access and barriers are correct for ARM?
<clever> 2022-05-18 19:21:08 < geist> random things i found while refreshing my cache of ARM docs on arms site:
<clever> 2022-05-18 19:21:38 < geist> https://developer.arm.com/documentation/100940/0101/?lang=en is a pdf that seems to try to summarize armv8 address translation, may be a handy read for folks trying to learn
<bslsk05> ​developer.arm.com: Documentation – Arm Developer
<clever> 2022-05-18 19:22:17 < geist> https://developer.arm.com/documentation/100941/0101?lang=en is a pretty good overview of some of the memory model stuff and barriers and whatnot
<bslsk05> ​developer.arm.com: Documentation – Arm Developer
<clever> mrvn: these 2?
bradd has quit [Quit: No Ping reply in 180 seconds.]
<mrvn> That's for 64bit. Not sure that covers 32bit as well.
<mrvn> And no, that just explains what opcodes exist. Not how you should use them
bradd has joined #osdev
<clever> i would expect the 32bit docs to have a similar title, try searching for that?
<bslsk05> ​developer.arm.com: Documentation – Arm Developer
<clever> > Instruction Synchronization Barrier(ISB) is used to guarantee that any subsequent instructions are fetched, so that privilege and access are checked with the current MMU configuration. It is used to ensure any previously executed context-changing operations, such as writes to system control registers, have completed by the time the ISB completes.
<clever> mrvn: is that not enough of a "when to use"?
<mrvn> There was some Guide that gave examples and explained why they failed and what barriers to use to fix it.
<clever> ah
<clever> skimming over the last link i pasted, my rough directions are to isb any time your changing mmu state
sikkiladho_ has quit [Quit: Connection closed for inactivity]
<clever> dmb any time your dealing with things like a mutex or xhci, and need the writes to be visible in a certain order
<clever> and dsb if your doing something like wanting to sleep AFTER an mmio write to the power controlled went thru
<clever> mrvn: one of the docs i read a while back, mentioned that if every core is sleeping (wfi/wfe), the arm core sends a special signal out, and the implementor can rig that into power saving logic
<clever> so basically, you first write to an MMIO to tell external hw that you want to enter a deep sleep, then once every core wfe's, the external hw is informed of that, and you can cut power to things
<mrvn> clever: ARMv6 might not even have dmb/dsb. those coprocessor commands are even harder to understand.
<clever> and that seems like a perfect use-base of dsb, so you dont sleep too early
<clever> yeah, i still need to figure out v6
<clever> my LK-armv6 port is very hacky, every barrier is a no-op, and the mmu is off
<clever> but i do have non-lk v6 mmu working
<clever> if you want to take a peek
<mrvn> The example I remember is this: Thread1: flag1 = 1; while(flag2 == 0) { } Thread2: flag2 = 1; while(flag1 == 0) { } this might deadlock forever.
<mrvn> neither write becomes observable to the other core.
<clever> that could even deadlock on x86, if gcc re-orders things
<clever> there are also compiler level barriers, to prevent gcc from reordering
<mrvn> flags have to be volatile so no reorder by the compiler.
<clever> if flag1/flag2 are not volatile, the store can be moved around
<clever> ah
<bslsk05> ​github.com: rpi-open-firmware/mmu_asm.s at master · librerpi/rpi-open-firmware · GitHub
<clever> here is an armv6 dsb opcode, when turning the mmu off
<clever> it also occurs in the dcache disable, up on line 61
<bslsk05> ​github.com: rpi-open-firmware/mmu.c at master · librerpi/rpi-open-firmware · GitHub
<clever> which gets used here
<clever> and now i notice, i never turned off the dcache
<mrvn> I wonder if this can't also happen on x86_64 because the while could be reordered and the whole loop runs in microcode over and over.
<clever> and armv6 did have a lot of wonky bugs booting linux.....
<mrvn> reordered in the cpu due ot out-of-order execution
<clever> i need to revive this bootloader, and you may have just answered the question of why its bork
<clever> where do i enable the dcache?...
<clever> i dont think v6 ever turned it on
<clever> maybe thats why it was fine
<mrvn> having a 10x slowdown due to uncached memory might be ok in a bootloader.
<mrvn> 100x slowdown without i-cache not so much
<clever> i did observe a noticable boost to SD read perf by turning the d-cache on
<clever> but that was probably on pi3 i think
<clever> the pi1 also lacks an arm L2 cache
<clever> i think it only has 16kb of i-cache and 16kb of d-cache, both L1
<mrvn> The specs don't say, you have to read some config registers to know.
<bslsk05> ​github.com: rpi-open-firmware/mmu.c at master · librerpi/rpi-open-firmware · GitHub
<clever> mrvn: this dumps those registers
<bslsk05> ​github.com: rpi-open-firmware/arm-arch.txt at master · librerpi/rpi-open-firmware · GitHub
<clever> and the output of that code, along with how i interpreted it
fwg has joined #osdev
<clever> the raw data is present, so you can decode it yourself, incase i made any mistakes
<mrvn> I think the only PI1 I have left is running my managed power outlet and I'm afraid of rebooting that as it might reset the relays and power cycle all my computers.
<clever> lol
<clever> the default pull direction can potentially cause that, depending on if its active low or high
<clever> the pi-zero also has an armv6
GeDaMo has joined #osdev
poprostumieciek has joined #osdev
opal has quit [Ping timeout: 268 seconds]
poprostumieciek has quit [Client Quit]
Vercas2 has joined #osdev
Vercas has quit [Quit: Ping timeout (120 seconds)]
Vercas2 is now known as Vercas
opal has joined #osdev
fwg has quit [Quit: .oO( zzZzZzz ...]
fwg has joined #osdev
opal has quit [Remote host closed the connection]
Vercas has quit [Remote host closed the connection]
the_lanetly_052 has joined #osdev
opal has joined #osdev
Vercas has joined #osdev
the_lanetly_052_ has quit [Ping timeout: 244 seconds]
Celelibi has quit [Ping timeout: 240 seconds]
nyah has joined #osdev
fwg has quit [Quit: .oO( zzZzZzz ...]
fwg has joined #osdev
fwg has quit [Client Quit]
j`ey has quit [Remote host closed the connection]
j`ey has joined #osdev
mzxtuelkl has joined #osdev
gog` has joined #osdev
justmatt has joined #osdev
orccoin has joined #osdev
ptrc has quit [Remote host closed the connection]
ptrc has joined #osdev
Celelibi has joined #osdev
fwg has joined #osdev
gxt has joined #osdev
fwg has quit [Quit: .oO( zzZzZzz ...]
irl25519 has joined #osdev
opal has quit [Ping timeout: 268 seconds]
gxt has quit [Ping timeout: 268 seconds]
gxt has joined #osdev
opal has joined #osdev
pretty_dumm_guy has joined #osdev
irl25519 has quit [Quit: irl25519]
irl25519 has joined #osdev
irl25519 has quit [Quit: irl25519]
irl25519 has joined #osdev
<zid`> clever: I need a pi nerd, someone's asking if you can make any of the armv7 opcodes act like their armv4t equivalents so that he doesn't have to trap them all for old software
<zid`> like through cp15
irl25519 has quit [Quit: irl25519]
irl25519 has joined #osdev
<mrvn> old sofware or old hardware?
irl25519 has quit [Quit: irl25519]
<mrvn> old software shouldn't have any qrmv7 opcodes.
fwg has joined #osdev
<clever> zid`: ive not been paying that much attention to the arm side of the pi's, arm has fairly good docs, so there isnt that much to uncover
<clever> its more about properly parsing those docs for me
<zid`> okay yea that's what he was putting off doing himself, he said he just wondered if I knew before he dove in
<zid`> I'll let him know to just dive
irl25519 has joined #osdev
<clever> i think the most advanced arm thing ive had to deal with, is setting up ACTLR and the mmu
<clever> because i'm basically replacing a missing boot rom
<clever> what did cp15 mainly handle?
<zid`> hey don't ask me
<clever> heh
<zid`> I'm just the pigeon
<clever> arm has too many co-processors
<clever> aarch64 cleaned it up some
irl25519 has quit [Remote host closed the connection]
<clever> ah here is it, found a whole section on differences between armv4, v5, in the v7 docs
<clever> one sec
irl25519 has joined #osdev
<clever> zid`: pass this link along, and have him check out page 2573, and 2599 i think
<clever> > This appendix describes how the ARMv4 and ARMv5 architectures differ from the ARMv6 and ARMv7 architectures. It contains the following sections:
irl25519 has quit [Remote host closed the connection]
irl25519 has joined #osdev
foudfou_ has joined #osdev
fwg has quit [Quit: .oO( zzZzZzz ...]
irl25519 has quit [Quit: irl25519]
foudfou has quit [Ping timeout: 268 seconds]
blockhead has quit []
foudfou_ has quit [Remote host closed the connection]
foudfou has joined #osdev
irl25519 has joined #osdev
fwg has joined #osdev
foudfou has quit [Remote host closed the connection]
foudfou has joined #osdev
irl25519 has quit [Quit: irl25519]
irl25519 has joined #osdev
irl25519 has quit [Client Quit]
irl25519 has joined #osdev
irl25519 has quit [Client Quit]
irl25519 has joined #osdev
[itchyjunk] has joined #osdev
irl25519 has quit [Quit: irl25519]
opal has quit [Remote host closed the connection]
gxt has quit [Remote host closed the connection]
irl25519 has joined #osdev
opal has joined #osdev
fwg has quit [Quit: .oO( zzZzZzz ...]
gxt has joined #osdev
irl25519 has quit [Remote host closed the connection]
<clever> mrvn, zid`: heard of the new #embed? https://thephd.dev/finally-embed-in-c23
<bslsk05> ​thephd.dev: finally. #embed | The Pasture
gxt has quit [Ping timeout: 268 seconds]
opal has quit [Ping timeout: 268 seconds]
Vercas has quit [Quit: Ping timeout (120 seconds)]
Vercas has joined #osdev
irl25519 has joined #osdev
irl25519 has quit [Quit: irl25519]
irl25519 has joined #osdev
irl25519 has quit [Quit: irl25519]
irl25519 has joined #osdev
opal has joined #osdev
irl25519 has quit [Remote host closed the connection]
irl25519 has joined #osdev
gxt has joined #osdev
irl25519 has quit [Quit: irl25519]
lg has quit [Ping timeout: 255 seconds]
irl25519 has joined #osdev
lg has joined #osdev
irl25519 has quit [Quit: irl25519]
irl25519 has joined #osdev
irl25519 has quit [Quit: irl25519]
irl25519 has joined #osdev
irl25519 has quit [Quit: irl25519]
irl25519 has joined #osdev
irl25519 has quit [Quit: irl25519]
irl25519 has joined #osdev
sikkiladho_ has joined #osdev
irl25519 has quit [Quit: irl25519]
irl25519 has joined #osdev
fwg has joined #osdev
irl25519 has quit [Quit: irl25519]
irl25519 has joined #osdev
nullndvoid has quit [Ping timeout: 255 seconds]
irl25519 has quit [Quit: irl25519]
irl25519 has joined #osdev
irl25519 has quit [Quit: irl25519]
irl25519 has joined #osdev
fwg has quit [Quit: .oO( zzZzZzz ...]
irl25519 has quit [Quit: irl25519]
[itchyjunk] has quit [Read error: Connection reset by peer]
irl25519 has joined #osdev
CompanionCube has quit [Quit: ZNC - http://znc.in]
Stary has quit [Quit: ZNC - http://znc.in]
heat has joined #osdev
mzxtuelkl has quit [Quit: Leaving]
irl25519 has quit [Quit: irl25519]
irl25519 has joined #osdev
irl25519 has quit [Quit: irl25519]
dude12312414 has joined #osdev
irl25519 has joined #osdev
irl25519 has quit [Remote host closed the connection]
irl25519_ has joined #osdev
blockhead has joined #osdev
irl25519_ is now known as irl25519
h4zel has quit [Ping timeout: 268 seconds]
the_lanetly_052 has quit [Ping timeout: 272 seconds]
elastic_dog has quit [Ping timeout: 264 seconds]
elastic_dog has joined #osdev
irl25519 has quit [Quit: irl25519]
irl25519 has joined #osdev
gxt_ has joined #osdev
gxt has quit [Remote host closed the connection]
Reinhilde is now known as AmyMalik
Stary has joined #osdev
irl25519 has quit [Quit: irl25519]
h4zel has joined #osdev
irl25519 has joined #osdev
fwg has joined #osdev
fwg has quit [Client Quit]
Stary has quit [Ping timeout: 268 seconds]
tsraoien has joined #osdev
CompanionCube has joined #osdev
dude12312414 has quit [Quit: THE RAM IS TOO DAMN HIGH]
orccoin has quit [Ping timeout: 276 seconds]
irl25519 has quit [Quit: irl25519]
irl25519 has joined #osdev
vdamewood has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
Stary has joined #osdev
bradd has quit [Ping timeout: 272 seconds]
bradd has joined #osdev
gxt_ has quit [Remote host closed the connection]
gxt_ has joined #osdev
irl25519 has quit [Quit: irl25519]
irl25519 has joined #osdev
irl25519 has quit [Quit: irl25519]
irl25519 has joined #osdev
irl25519 has quit [Quit: irl25519]
irl25519 has joined #osdev
fnoam has joined #osdev
fnoam has quit [Max SendQ exceeded]
irl25519 has quit [Quit: irl25519]
irl25519 has joined #osdev
fnoam has joined #osdev
fnoam has quit [Max SendQ exceeded]
<geist> ah good ol armv4 vs v5 vs v6
<geist> i used to know that like the back of my hand
<geist> especially all the thumb interworking instruction shenanigans and limitations you had back then
<geist> thankfully most of it became irrelevant in v6, and then v7 added thumb2 which doubleplus so made it irrelevant
irl25519 has quit [Quit: irl25519]
fwg has joined #osdev
sikkiladho_ has quit [Quit: Connection closed for inactivity]
mjg has joined #osdev
h4zel has quit [Ping timeout: 268 seconds]
fwg has quit [Quit: .oO( zzZzZzz ...]
Vercas has quit [Quit: Ping timeout (120 seconds)]
Vercas has joined #osdev
<heat> sup
<zid`> hi boils
<heat> guys, how to develop opearating systm???
<zid`> Do you know assembly, C, gcc, binutils, gnu make, a shell, qemu, etc?
<gog`> no
<zid`> Perfect, just like everybody else who asks that question
<zid`> come back in 15 years
<GeDaMo> Then you won't have to unlearn :P
<gog`> nice
<gog`> also i will never learn
<gog`> jsyk
<danlarkin> Hi everyone, I'm developing an operating system in HTML and CSS. So far I have it printing out Hello World in firefox, what's next?
<psykose> a ravenous heroin addiction
<zid`> DanDan: javascript, probably?
<j`ey> hello world in Chrome
<GeDaMo> A Javascript interpreter :|
<gog`> when you're writing a dialect of python and its interpreter
<fluix> oh psykose you're here, o/
<gog`> then you know you're in deep
<psykose> maybe i'm secretly a ghost
<zid`> I should write more of my C compiler
<gog`> i should work on anything at all
<fluix> not so secret now!
<fluix> not so secret now!
<zid`> gog` help me
<gog`> ok
<zid`> how do I add functions and shit to my precedence climber
<gog`> why do you want to add shit to it
<zid`> so that I can parze more than expressions
<zid`> do I have to just wrap shit around it, or can it be extended to also include things that aren't normal mathy expressions?
<zid`> (like recursive descent)
<psykose> i think functions start with a declaration like `function`, that would be a good starting point imo
<zid`> can't tell if trying to help and bad, or funny
<zid`> cus I am also bad
<psykose> naughty lad
<zid`> I probably know a parser nerd somewhere I can ask, if I can manage to explain things in idiot terms to them while they're busy trying to think in context free grammars and LR and shit
<zid`> gotta cross that language gap
<GeDaMo> I only understand recursive descent :|
<gog`> i know literally nothing about this and i will not learn
<GeDaMo> "PEG-based transformer provides front-, middle and back-end stages in a simple compiler" http://www.vpri.org/pdf/tr2010003_PEG.pdf
<zid`> I'm not sure I am into pegging
<zid`> I've never tried it
<zid`> also this is for lisp so it makes for an awful example parser cus it does nothing interesting :p
<psykose> i heard it's pretty fun u should try
GeDaMo has quit [Quit: There is as yet insufficient data for a meaningful answer.]
<moon-child> how is OPERAT formed
<klys> operations errata
<heat> operat systen
fwg has joined #osdev
<jjuran> An operetta about op errata
fwg has quit [Client Quit]
<gog`> they need to do way instain programmer
<heat> can i write os in rust?
<zid`> no
<gog`> mostly
<zid`> maybe someone good at programming could though
<gog`> leaves me out
<psykose> could jeff bezos write an os in rust
<gog`> nah he'd pay somebody to
<heat> j`ey can
<gog`> he doesn't have time to waste on something that pointless
<heat> he is CEO of rust after all
<j`ey> hello i can provide rust OS for a fee
<gog`> how much
jjuran has quit [Quit: Killing Colloquy first, before it kills me…]
<j`ey> 1 million dollars
<zid`> I will provide one for $800 a line
jjuran has joined #osdev
<heat> no
<gog`> leaves me out
<heat> what about
<heat> 15 canadian dollars
<zid`> You can have 1/8th of a line
<moon-child> I'll do it for $600
<moon-child> but every line begins with the characters //
<heat> can i delete them or is it illegal
<zid`> Can I write it in C for $40?
<heat> not interest
<moon-child> no
<moon-child> unsafe
<moon-child> go to rust jail
<heat> i have big idea for operating
<heat> modern, safe
<heat> C bad old
<klys> go on
<zid`> https://godbolt.org/z/xGWdPfzaj Here's the full source
<bslsk05> ​godbolt.org: Compiler Explorer
<heat> join me, i have idea, you write program, we success
<jjuran> heat: Now that you have the idea, you just need a technical co-founder... and you beat me to it :-)
<klys> what when compiler give error
<zid`> heat: I'd say it's worth more than $40 personally
<heat> is unsafe
<heat> not interest
<heat> as said before
<klys> is idea rust compiled
<klys> so many know rust not me
<klys> how to get ahead
<moon-child> klys: supposedly, bdsm-compiler will lightly spank you and tell you how to fix your code
<klys> badsim
<klys> bin for adverting similarities
<klys> companion to the practical extraction and rubbish lister
<klys> when you rubbish list the bin you get unique talent
poprostumieciek has joined #osdev
poprostumieciek has quit [Client Quit]
opal has quit [Remote host closed the connection]
<mrvn> clever: so #embed turns a binary file into hex?
[itchyjunk] has joined #osdev
opal has joined #osdev
<mrvn> Somehow I feel that #embed is kind of wasted. It's not that hard to turn a binary into hex and #include it. But the c++ proposal of `std::embed` sounds like a whole different beast.
<mrvn> constexpr std::span<const char> art_data
<mrvn> = std::embed("/dev/urandom", 32);
<mrvn> Gets kind of scary when your builds aren't reproducible though. What's next? std::embed_popen("./run-my-script");?
<geist> oh wow
h4zel has joined #osdev
<geist> not reproducible is one thing, but causing a compiler to run some arbitrary code is a bit iffy
<geist> though i guess the latter you just made up.
<mrvn> geist: note that is not proposed (yet)
<mrvn> I think proposing to embed the output of an arbitrary command would make their heads explode.
<mrvn> The proposal only has `std::embed(filename)` and `std::embed(filename, size)`
<mrvn> Kind of think `std::embed(filename, size, start_offset)` should be a thing.
<mrvn> But it looks like it's supposed to be a view of the file contents so slicing it should be efficient.
<heat> I think that's a whole lot of overengineering
<heat> #embed is fine
<mrvn> If/When gcc/clang get std::embed I will definetly use it. E.g. to include the font data in the kernel.
<mrvn> heat: But what is the point? The cpp turns the binary into hex, the compiler then has to parse that back into binary. What exactly is the advantage over #include?
<heat> #include can't include binary files
<mrvn> And concatenation is ugly since #embed lacks the trailing ","
<mrvn> heat: hexdump the binary into a header, include header.
<heat> and that's what you're trying to solve
<heat> skip the hexdump step
<heat> why else would you need #embed?
<mrvn> heat: imho that's not worth the extra work for the compiler
<heat> or std::embed, or whatever
<heat> just run a separate build step
<mrvn> std::embed is different and the compiler (as opposed to the preprocessor) can actually do something with ti.
poprostumieciek has joined #osdev
<mrvn> Most importantly don't hexdump the binary just to parse it back into binary.
poprostumieciek has quit [Client Quit]
<heat> that inhibits some optimizations as well
<mrvn> like?
<heat> const uint8_t config_data = [std::embed("config.data")];
<heat> if (config_data[0] == 0x10) //be verbose
<mrvn> heat: it's a constexpr so you can even do "if constexpr (config_data[0] == 0x10) //be verbose"
justmatt is now known as justmatt_
justmatt_ is now known as justmatt
<heat> if its a constexpr, the compiler can't just turn it into binary
<heat> it will need an internal representation of it
<gog`> just add it with a linker script to .data
<mrvn> heat: it's a blob of std::byte
<mrvn> constexpr std::span<std::byte> foo = std::embed("foo");
<heat> #embed is way smarter than "dump it into text"
<mrvn> heat: if you merge the preprocessor and compiler pass.
<mrvn> So now you have to implement it twice.
<heat> no
<heat> it is way smarter than dump it into text
<heat> I'm fairly sure it isn't being done as a preprocessor step
<bslsk05> ​www.open-std.org: N3017: #embed - a scannable, tooling-friendly binary resource inclusion mechanism
<heat> the benchmarks
<mrvn> Are you saying I can't "gcc -E" it and see the hexdump?
<mrvn> Can use it in .S files?
<heat> I don't know
<heat> but it's way faster than a naiive text-based approach
<heat> like you'd expect if you generated an array of uint8
<mrvn> Yeah, there must be some magic going on there that skips the hexdump part.
<bslsk05> ​www.open-std.org: N3017: #embed - a scannable, tooling-friendly binary resource inclusion mechanism
<mrvn> With that optimization can I even do: const char data[] = {#embed <header.bin> , #embed <body.bin> };?
<heat> *shrug*
Vercas has quit [Remote host closed the connection]
Vercas has joined #osdev
<mrvn> Ohh, vendor extensions on #embed are a nice touch: #embed </dev/urandom> limit(2+2) sdcc::element_type(int32_t)
<mrvn> Is that in C23 too? Can that be used on any preprocessor directive?
<moon-child> I mean, presumably you wouldn't use it for #define
<moon-child> or #if
<mrvn> #define would be tricky as "x::y" occurs naturally. #if BLA = 0 x::y should parse.
<mrvn> #include <math.h> gnu::fast_math(true)
<mrvn> I will probably always think of "#embed" as turning the file into hex and including it but still looking forward to using it.
<mrvn> I wonder if this works without "limit": static const char blob[16] = { #embed </dev/urandom> };
<moon-child> probably hangs
<mrvn> For a fixed size array the compiler could has "limit(16)" implicitly.
<moon-child> I guess
<zid`> you know what it's not faster than? bin2o :P
<heat> i think it should time out/exaust resources
<heat> have a default byte limit for embed operations
<zid`> that proposal's been in limbo for years, my thought on it was always "too little too late bruh"
<zid`> we've had bin2o for 25 years
<mrvn> zid`: you can't do static_assert on bin2o
jack_rabbit has quit [Quit: ZNC 1.8.2 - https://znc.in]
<mrvn> heat: Without the fixed array it definetly fails after a long time when memory runs out
knusbaum has joined #osdev
<mrvn> Didn't godbolt used to have the circle compiler?
vdamewood has joined #osdev
<geist> heat: hmm, going through old pull requests: is there *any* downside you know of to the 'use gcc as linker' CL?
<geist> it seems to work fine, but i'm still nervous about pulling that trigger
<geist> maybe it'll conflict with using clang as a compiler (which is another CL from someone else)
<mrvn> geist: what if you compile with clang? Or is that a use CC to link?
<geist> use CC to link
<mrvn> That's what I always used before I had to fiddle with LTO for temp libraries.
<geist> for historical reasons the LK build system uses ld directly but for LTO purposes and whatnot heat has a PR to switch that to using cc
<mrvn> LTO is definetly simpler if you can use CC
<geist> which seems to work fine, aside from needing to append -Wl and whatnot and a bunch of extra -freestanding and whatnot switches to the linker
<mrvn> translating linker options to CC options can be a pain.
<mrvn> personally I would look into the clang PR first and then check if linking with clang works as well
<geist> yeah, that's probably a good idea. most likely they're compatible, but indeed
<geist> the clang one i'm more worried about, since it's harder for me to control what clang is used, etc. the gcc stuff is fairly straightforward (or at least i know it better)
<geist> ie, if a gcc for a triple exists it probably will do what i want
<mrvn> on the other hand using a different compiler can show you bugs in your code.
<geist> clang is more like 'does this version have the bits i need' so might need to probe it more
<geist> and of course i have to support both toolchains since LK runs on arches that clang doesn't compile for
<geist> oh totally. 100%
<mrvn> different warnings, different optimizer and analyzer, different code gen-
<geist> for that reason we still build the zircon kernel with GCC, even if we dont use it
<mrvn> last one can be a big problem if you have sketchy code that will suddenly break
<geist> between clang and gcc and warnings cranked to the hilt, we catch lots of stuff at build time
<mrvn> can you sue thge clang sanitizers in the kernel
<mrvn> s/sue/use/
<mrvn> ?
<geist> a fair amount of them yes
<geist> though depends precisely on what yo mean by sanitizer
<heat> geist, microblaze was iffy
<geist> yeah i saw that. but it does build and run, so *shrug*
<heat> but apart from that, no, not really
<geist> it's a tier 3 arch if at best
<mrvn> like address sanitizer to find buffer overflows. Do you have to provide a hook for how it reports errors?
<heat> you need to implement the sanitizer itself
<geist> falls into the category of 'as long as it compiles i'm okay, file a bug and move on to fix it'
<geist> ah, yes we use KASAN for example
<geist> and we just added code coverage stuff
RAMIII has quit [Quit: WeeChat 3.5]
<heat> you can't use most sanitizer implementations clang has, you can only use the codegen part of the sanitizer
<mrvn> any idea how to do code coverage on constexpr?
<geist> (i'm talkinga bot zircon. LK does none f these in mainline branch of course)
<mrvn> heat: anything that uses shadow ram would need extra work, obviously.
<geist> i know for example that the tons of places within google that use LK they almost all use clang, so the clang patch has been done before, and some of them i think use ASANs and whatnot
<geist> however, in 100% of those branches they made the change in a non compatible way
<geist> usually for the few arches they want
<heat> mrvn, how it works is that the codegen part expects a few callbacks
<heat> i.e __asan_load_8 for a 8 byte load
<heat> you implement those, clang implements those, the linux kernel implements those, etc
<mrvn> heat: does ASAN change the representation of pointers?
<heat> no
<geist> yep. the memory management of the shadow bits gets tricky but it's fairly straightforward in user space
<mrvn> So where does it get the buffer size from?
<heat> it doesn't
<heat> shadow bitmap
<heat> anyway, asan/kasan is a special boy
<geist> right, the implementation maintains a shadow bitmap somewhere somehow
<heat> the compiler can inline asan checks
<geist> code coverage for example allocates a large buffer in bss that is a bunch of counters for regions of code
<mrvn> The signed integer overflow check should be straight forward.
<geist> and the codegen is basically a bunch of global incs as each block is entered
<mrvn> geist: hope it's saturating
<heat> this means that you either disable inline asan instrumentation, or you do it as they want to (flat shadow bitmap with everything mapped)
<geist> 4 bil or so, so probably doesn't matter
<mrvn> geist: you would have to use something exactly 2^32 times to reset to 0, unlikely.
<geist> no doy
<geist> heat: hrm, so used to gerrit. i rebased and fixed your change locally, but there's no way for me to easily update the PR
<geist> gerrit i'd just tell it to rebase and re-upload
<mrvn> What do you have to provide so the sanitizers can report an error?
<heat> you should look at docs mrvn
<heat> each sanitizer is different
<mrvn> I should
<heat> since the docs are kinda crap, look at docs and actual implementations
<heat> geist, can you git format-patch it and give me a pastebin?
<geist> oh sure
<mrvn> using them inside kernels is a rather small subset. Not something the docs are written for.
<heat> it's the same
<heat> kasan is indeed a smaller subset, but it's also technically a separate sanitizer
<heat> the rest (ubsan, tsan, sancov) are the same
<heat> msan too
<mrvn> heat: normaly you can just fprintf(stderr, ) errors and exit()
<heat> geist, wait format-patch won't work here
<heat> I'll rebase it here and push
<heat> mrvn, that's very high level
<geist> sure it does, i already cherry picked it and fixed it
<heat> not something the sanitizer does
<geist> well, either way. the only real diff is in the arch/riscv/rules.mk
<mrvn> heat: yes. But that's what I was asking about. If there is some generic stuff you have to provide that all sanitizers than use.
<heat> no
<mrvn> "There is a minimal UBSan runtime available suitable for use in production environments. This runtime has a small attack surface. It only provides very basic issue logging and deduplication, and does not support -fsanitize=function and -fsanitize=vptr checking.
<mrvn> To use the minimal runtime, add -fsanitize-minimal-runtime to the clang command line options. For example, if you’re used to compiling with -fsanitize=undefined, you could enable the minimal runtime with -fsanitize=undefined -fsanitize-minimal-runtime." That sounds like a good start.
<heat> thank mr geist
<geist> oh hrm,, i just noticed the extraneous 'comma' in build.mk. do need that?
<heat> apparently not
<heat> I remember I messed around with it
<geist> if so there's a COMMA already declared in make/macros.mk anyway
<heat> updated the PR
<geist> rad
<heat> deg
<geist> grad
justmatt has quit [Quit: see you tomorrow!]
<geist> hmm, jut stumbled upon this paste.c-net.org from google but it seems decent
<geist> i was dissapoint to discover that gist.github doesn't suppotr file uploads, or at least not in a way i know.
<heat> I usually use termbin
<heat> I think github has a command line tool?
<zid`> geist: you can use github.io for that though technically?
<geist> yah probably so. i bet you can do some sort of push via magic to github
<heat> gh gist create -d "my test gist" -f some_local_file.txt test_gist from their tool
<geist> ah yeah they have some github cli. i dont use any of that, so yeah
janemba has quit [Ping timeout: 260 seconds]
<geist> though i guess i should at least look to see if there are valuable things there
<heat> like 60% of my OS's userspace build time is dominated by two google libraries
<heat> this makes me uncomfortable
<zid`> Sounds normal
<heat> late stage-cplusplusness
<zid`> dw, it'll be 100% soon
<zid`> also you'll need to swear your allegiance to the flag of google
<zid`> and provide a verification blood sample
<heat> when does the goog take over my project?
<zid`> and tell it one of your deepest secrets
<zid`> never, you just become part of the google hivemind
<heat> subscribe
<zid`> more and more of your life just becomes doing menial tasks for google's AI
<zid`> solving captchas, identifying bicycles
<zid`> in exchange, they dont' cut you off from society
<heat> toybox's bc implementation (in C) takes 5.5s to compile
<heat> woah
<zid`> too long
<heat> I know
<heat> it's impressive
<zid`> are you going to make a nice html report
<zid`> with pie charts
<zid`> and hovers for breakdowns
h4zel has quit [Ping timeout: 240 seconds]
<heat> i have one
<heat> ui.perfetto.dev
<heat> plus .ninja_log
<heat> turns out the hardcore C++ google writes can also be very useful
<heat> this plus clang's traces means you can get a fantastic insight into your build
<heat> I should open a fuchsia trace
<geist> okay, this clang patch is totally incompatible
<geist> i have no idea what version they have, but it leaks all kinda host bits into things, and then tries to use lld directly
<heat> lol
<geist> ah, clang as a linker seems to keep trying to use the host version
<geist> even with -fuse-ld=lld
<heat> host version of what?
<geist> the linker. sorry, it falls through to /usr/... and then tries to use whatever linker is there
<geist> and thus gets confused becayse arm
<heat> I think that's natural since the patch uses a clang installed on /usr
<geist> yah
<heat> and that *should* be fine
<geist> well the default patch tries to use ld.lld, which works standlaone. but not with your 'use the compiler as the linker' patch
<heat> ld.lld at least supports all architectures AFAIK
<geist> so lemme locally revert yours first and then try again
<heat> even with -fuse-ld=lld?
<geist> yes even weith
<geist> it seems to just straight up ignore that. i've run it locally with -v and clang just slams directly through to gcc/lld
<geist> and i do have an lld installed locally
<geist> er gcc/ld
<geist> basically `clang --target=aarch64-elf -fuse-ld=lld -r foo.o bar.o -o foo.bar.o` (this is for the incremental link stuff) it just immediately falls through to collect2 from gcc which then uses bfd ld
<geist> even without -r it still just ignores it
<geist> though it doesn't completely ignore it, it requires that you put something valid in -fuse-ld=*
<heat> it works here
<geist> maybe there's some sort of ubuntu/mint linux/etc override that says 'always use bfd linker'
<bslsk05> ​gist.github.com: gist:671702f95533817010f815560773b64d · GitHub