<clever>
dolphinana: with this, qemu should be running rpi1-test, its listening for gdb, and the cpu is paused immediately out of reset, so gdb can see everything....
<dolphinana>
I'm back :)
<clever>
xp /fmt addr -- physical memory dump starting at 'addr'
<dolphinana>
Should I use qemu?
<clever>
this is from `help` in the qemu monitor console (a tab in the gui)
<clever>
i think i'll find the answer to the problem on this end, but you can follow along if you want to see how it all works
<clever>
because i have this in my copy, from when i was last debugging this issue
<dolphinana>
you said cast? Does this have anything to do with pointers?
<clever>
yeah
<clever>
(gdb) print $r0
<clever>
$1 = 119188
<clever>
without the cast, gdb has no clue what the number means
<clever>
because all the debug info (the elf) isnt loaded
<dolphinana>
where's the debug info?
<clever>
in the .elf file
<clever>
but gdb is having trouble loading it
<clever>
so i just didnt bother loading it
<clever>
since i dont care about the printf, i went back to unmask_interrupt, and i can see 454 is where it calls printf, and 458 is right after printf has returned
<clever>
(gdb) break *0x458
<clever>
(gdb) cont
<clever>
Breakpoint 2, 0x00000458 in ?? ()
<clever>
so i can set a breakpoint right there, continue, and then go back so stepi'ing
<clever>
reg==0x2000b214, bit==25
<clever>
this also popped out on serial0
<dolphinana>
stepi'ing sounds goofy :o
<clever>
468: eb00637c bl 19260 <puts>
<clever>
46c: eaffffdb b 3e0 <unmask_interrupt+0x84>
<clever>
puts("unmasked")
<clever>
the printf spam i added ages ago, when i had the same problem
<clever>
(gdb) break *0x46c
<clever>
Breakpoint 3, 0x0000046c in ?? ()
<clever>
so ive skipped ahead again
<clever>
408: f1080080 cpsie i
<clever>
and it hits this opcode, which i believe unmasks the irq
<dolphinana>
What does "unmasking the irq" mean?
<dolphinana>
and what does it mean when irq is masked?
<clever>
while holding a spinlock, all irq's are masked (blocked)
<clever>
so the code cant be interrupted by things
<dolphinana>
so the opcode we're talking about now: all irq's would be unmasked (unblocked)?
<clever>
yeah
<clever>
(gdb) help stepi
<clever>
Usage: stepi [N]
<clever>
Argument N means step N times (or till program stops for another reason).
<clever>
its not failing the smae way as hardware, so i'm going to try a more brute-force solution
<clever>
(gdb) stepi 1000
<clever>
0x00000e98 in ?? ()
<clever>
this will just run 1000 opcodes
<clever>
and then i just keep doing that, until it boot-loops.....
<dolphinana>
Can you pass 0xff as argument to stepi?
<dolphinana>
I mean,
<dolphinana>
Can you pass value in hexadecimal form?
<clever>
probably
<dolphinana>
Just a random thoughts in my head
<clever>
0x0001cbc8 in ?? ()
<clever>
(gdb) stepi 10000
<clever>
(gdb) stepi 10000
<clever>
0x0001b7cc in ?? ()
<clever>
0x000011cc in ?? ()
<clever>
1k was a bit slow, so i switched to 10k
<clever>
cpu 0 data abort, synchronous external abort on read
<clever>
DFAR 0x2040000c (fault address)
<clever>
halting
<clever>
DFSR 0x8 (fault status register)
<clever>
and it then printed this in the serial0
<clever>
so, lets assume 0x0001b7cc was it working "normally", and then 0x000011cc is after it failed
<clever>
and it ran 10k opcodes in between those 2
<clever>
1b7cc is within __debug_stdio_write, that doesnt help much....
<clever>
oh
<clever>
starting app loader
<clever>
but this is also the last thing it printed
<clever>
so i can set a breakpoint there
<clever>
loader_entry() then
<dolphinana>
where's the "starting app loader" address
<clever>
00003234 <loader_entry>:
<clever>
this address in the .lst file
<dolphinana>
I see
<clever>
app/linux-bootloader/loader.c here in the src
<dolphinana>
I found it
<clever>
(gdb) break *0x3234
<clever>
oh, but i want to reset it all
<dolphinana>
I found the function in app/linux-bootloader/loader.c :)
<clever>
so i'll close qemu and gdb, and then restart/reconnect
<dolphinana>
If I was the one running gdb, I would've entered `break *0x33a8` since the loader_entry function is at 0x000033a8
<clever>
i did stepi 100, 6 times, and now its within arm_data_abort_handler
<clever>
went too far!
<clever>
so now i'll restart, and `stepi 500` this time
<dolphinana>
do it
<dolphinana>
what were you going to do now again?
<clever>
loader_entry
<dolphinana>
step into the loader_entry with gdb or...?
<clever>
the rpi has a 64 counter, that counts up at 1mhz
<clever>
the ST_CLO and ST_CHI registers, give the lower and upper 32bits of that count
<clever>
then you have 4 compare registers, ST_C0 thru ST_C3
<clever>
and when a compare register matches ST_CLO, it will fire one of the 4 timer interrupts
<dolphinana>
wait... so...
<dolphinana>
Can you tell me more about "the rpi has a 64 counter, that counts up at 1mhz" ?
<clever>
there is just a 64bit number in the chip, that is counting up, 1 million times per second
<dolphinana>
so after one seconds from the beginning, that 64bit number in the chip would've counted to 1 million?
<clever>
yep
<dolphinana>
wait, what's the maximum value that can be stored in 64 bit?
<clever>
so if you want something to happen 0.5 seconds from now, you would get the current value of that counter, add 500k, then write that sum to a compare channel
<clever>
this was found on the bottom of a pi2 board i believe
<clever>
its the VPU jtag port
<clever>
and youll find that on nearly every model of pi
<clever>
3, 4, 400, zeros
<G33KatWork>
ahhh, nice. they labelled it on old chips
<clever>
but the pi5, has 2 of those headers
<clever>
somebody on the forums said there isnt enough pins wired up, and it may be SWD
<clever>
i also discovered on the forums, that the debug uart (between the 2 hdmi) can be remuxed to cortex-a76 SWD
<clever>
so you can debug the primary arm cluster via that debug header
<clever>
let me get the i2c read/write code...
<G33KatWork>
luke wren told stacksmashing (and he told me), that they are able to debug the RP1 over the PCIe link. it might have been altered along the way though and it's a misunderstanding
<G33KatWork>
but maybe they really have something in hardware in BAR0 to do this
<clever>
there are undocumented registers in the broadcom soc, that let you bit-bang jtag from the vpu
<clever>
i can picture how the same may exist on the rp1
<G33KatWork>
if you reset the RP1, the power LED turns orange because I guess the pin goes into high impedance. when I then try to reload it from linux, it stays that way
<clever>
ah
<G33KatWork>
then I wrote an endless spinning loop, loaded that, the i2c device vanished, so I knew that loading in theory worked
<clever>
one guess i had, is that the bcm2712 end of the pcie link also has to be reset
<G33KatWork>
and then I fought poking the right registers to turn on the LED I attached to GPIO17
<clever>
i guess the next goal, would be to try and turn on the RP1 pl011 uart?
<G33KatWork>
yeah
<G33KatWork>
should be easy, clocking is an issue though
<clever>
yeah, do you know the UUU trick?
<G33KatWork>
for that we have the linux clock driver though. But I don't think it contains all the clocks
<G33KatWork>
in the rp1 firmware is a lot of metadata structs about the clocking tree
<G33KatWork>
haven't reversed that yet
<clever>
yeah
<G33KatWork>
what's the UUU trick?
<G33KatWork>
autobauding?
<clever>
U in binary, is just 0101_0101
<clever>
plus the start and stop bits of uart, and it stays a perfect square wave
<G33KatWork>
I can just hook up an oscilloscope and measure the resulting bit timing and based off the divider figure out the input clock. but yeah, that works
<clever>
so you can just spit out U's endlessly, measure the frequency, half (or double?) that, and there is your baud
<clever>
U just makes it simpler, because there is no cell that is 2 bits wide
<clever>
it configures the clock dividers and PL011 from scratch
<clever>
so you can just steal the PL011 half
<G33KatWork>
oh, tip how I found the RP1 firmware without reversing the videcore code: I used cpu_rec on the ELF and it found some aarch64 and armhf code
<clever>
it also prints a U at the end to show signs of life, and give you something to trigger the scope on
<clever>
ahh
<clever>
ive not heard of cpu_rec before
<clever>
looks handy, just need to deal with the compression first
<G33KatWork>
the armhf code is the rp1 firmware. you need to go up a bit, because the vector table precedes the code, but you can spot the interrupt vectors all pointing to 0x20000000 and the initial stack pointer to somewhere in 0x10000000. that's the start
<G33KatWork>
cpu_rec runs standalone and as a binwalk plugin
<G33KatWork>
somebody recently reimplemted it in rust even
<G33KatWork>
oh, I haven't looked at that. only the RP1
<G33KatWork>
the RP2040 has some fifo mailbox thing for inter-core comms in the SIO block. that doesn't seem to exist on the RP1
<clever>
ive seen signs that the VPU is locked down far better now
<clever>
but the binaries ive seen, also seem to lack signatures
<clever>
how familiar are you with the boot chain on the VPU and its security?
<G33KatWork>
how does that work with secure boot? do you need to sign everything including firmware with your own key?
<G33KatWork>
I only know that they support secure boot, nothing more
<clever>
pi0-pi400 all support hmac-sha1 signatures
<clever>
a 20 byte key from the boot rom, gets XOR'd with a 16 byte key from OTP, to create a per-device key
<clever>
(rpf then ignores this, and programs every device within a model with the same key)
<clever>
that is then used to hmac-sha1 sign the first .bin stage
<clever>
on pi0-pi3, verification was never enabled, but the OTP half of the key is still burned in
<clever>
pi4 was the first model to enable the hmac check, but the early pi4 firmware didnt maintain the chain of trust
<G33KatWork>
do they always check the hmac on booting or only when secure boot is provisioned?
<clever>
pi4 always checks the hmac
<G33KatWork>
meh
<clever>
but the hmac isnt secure, i already dumped the key, and every device uses the same key
<G33KatWork>
oh right, it's symmetric
<clever>
the bcm2711C0 i believe added proper rsa signature support to the rom
<clever>
there are 4 public keys baked into the rom, and they can never be changed
<clever>
the .bin stage is then signed, with both the hmac and one rsa key
<clever>
an OTP flag will switch it over to rsa verification instead
<G33KatWork>
okay, so it can be made assymmetrical on request
<clever>
and if the .bin stage detects that RSA checking is enabled, it will forcibly set SIGNED_BOOT=1 in the SPI config file
<clever>
when SIGNED_BOOT=1 is set, there must be a pubkey.bin in the SPI, holding the user chosen rsa2048 key
<clever>
and bootconf.txt + boot.img must be sigend with that
<clever>
to prevent keychange attacks, the sha256 of pubkey.bin is burned into the OTP, and verified if RSA sigs are enabled
<clever>
so, you have 3 levels of security
<clever>
1: the default, just hmac
<clever>
2: set SIGNED_BOOT=1 in the spi config, include a key, and sign bootconf.txt + boot.img
<clever>
2 can be undone at any time, by just re-flashing the SPI to remove it
<clever>
3: burn the pubkey.bin hash to OTP and enable RSA sigs
<clever>
that key is now permanent, and you can never turn sig-checks off again
<G33KatWork>
right, so you'd need a code executing bug to get past this
<clever>
part of the support code for this, is that every compressed file in the SPI flash, has a sh255 footer, and the expected hash is baked into the bootcode.bin
<clever>
those hashes are checked, even when not doing secure-boot
<clever>
so if you try to modify bootmain.elf, then bootcode.bin will detect a corrupt hash and refuse
<clever>
and if you fix the hash in bootcode.bin, a pi4 will just not execute the .bin
<G33KatWork>
also the poke on address 0 in the firmware load routine seems weird
<clever>
yeah
<clever>
the rom will likely answer some of those questions
<G33KatWork>
0x40014000 is the reset block
<clever>
yeah
<clever>
which is undocumented
<G33KatWork>
a write at offset 0x3000 clears bits
<G33KatWork>
so they are clearing some reset bit
<G33KatWork>
that might explain why I couldn't read
<clever>
yeah
<G33KatWork>
let me boot the pi and try a few things
<clever>
my pi5 still hasnt shipped
<clever>
so ive been doing everything purely from the .bin files
<G33KatWork>
I was just lucky to know somebody who was in cambridge
<G33KatWork>
and then I got nerd sniped and tried things all weekend long while neglecting everything else :D
<clever>
heh
<clever>
on the signature side again (a bit), the pi4 stage1 files, all have "random" 20 bytes of garbage at the tail end
<clever>
that is the hmac-sha1 signature
<G33KatWork>
but now we are in a place where we could write some code to try and figure out which interrupt vectors and reset bits are for which peripheral and so on
<clever>
but the pi5 .bin files, differ in other places, but end in the same byte sequence
<clever>
implying its at least not signed the same way
<clever>
oh, and maybe some of these pokes, are required to enable some block, before you run the stock rp1.bin
<G33KatWork>
possible
<G33KatWork>
yep
<G33KatWork>
you need the reset poke
<clever>
aha!
<clever>
and maybe now you can run the stock rp1.bin as well?
<G33KatWork>
otherwise I get 0s
<clever>
G33KatWork: what about reading other things, can i2c now read the rom?
<G33KatWork>
original firmware is still very red
<G33KatWork>
and my blinky doesn't work. I think I am missing a reset as well
<G33KatWork>
I'm going to try...
<G33KatWork>
might just clear all the resets while I'm at it
<clever>
oh, and another thing, when you hit the watchdog register, it likely resets everything again
<clever>
and then the payload had to un-reset things
<G33KatWork>
It's cumbersome to try things as I am losing ethernet all the time
<clever>
yeah
<clever>
are you familair with ppp?
<G33KatWork>
yeah, could work
<G33KatWork>
or just xmodem
<clever>
oh, wifi?
<clever>
the wifi is into the soc i believe
<G33KatWork>
oh right
<clever>
and to reduce the fallout from killing the rp1, a dtoverlay could just disable the whole thing?
<clever>
maybe target the main rp1: node, so pcie is still there, and shows hotplug events
<G33KatWork>
I think the kernel drivers don't like detaching from devices during runtime right now
<G33KatWork>
I saw some todo comments
<G33KatWork>
and at least the adc driver causes a stack trace on unload
<clever>
but if the whole rp1 node is disabled in DT, the drivers never attach in the first place
<G33KatWork>
oh, yeah
<clever>
so it will act like a pcie device lacking drivers
<G33KatWork>
when loading it in config.txt
<G33KatWork>
yeah, I could build a more convenient setup
<clever>
do you have anything swd capable? another pi or pico?
<G33KatWork>
I even have the pico probe
<G33KatWork>
the uart cable fits nicely
<clever>
i think the pico-probe can also do SWD over the same pins
<clever>
which is perfect
<clever>
oh right, but we want the rp1 header, not the a76 header
<clever>
if you look at the bottom of the pi5, youll see an SMD near the usb port, and another near the camera ports
<clever>
if you probe those with a volt meter, do you see any activity?
<clever>
even just idle high would be a sign of life
<G33KatWork>
Copyright (c) Raspberry Pi (Trading) ltd. 2021