<clever>
the official headers that broadcom released years ago, only mention a single register, at that addr, and 32 bits in size
<bonda_000>
its a lie
<bonda_000>
references up to 7ea035xx
<clever>
yeah
<bonda_000>
there is a load of registers
<clever>
[clever@amd-nixos:~/apps/rpi/firmware]$ git show 4f63c3bdd4d9ca4f308e683fe7e70501b4ac3232
<clever>
in this commit of the official firmware, RPF messed up, and forgot to strip the binaryes
<clever>
so the .elf's in there, have all of the debug info still present
<clever>
somebody then complained that the files had doubled in size, and RPF fixed the issue :P
<clever>
bonda_000: with that, i can skip to the ISP area in ghidra, and see references from dozens of functions, isp_plat_init, isp_plat_stop, isp_plat_abort
<bonda_000>
yeah I don't see too many labels but there are some strings left in it
<clever>
if you checkout that version of the firmware, and load the old `start_x.elf`, youll get a lot more details
<bonda_000>
i can't, it says fatal bad object
<clever>
you need to `git fetch origin <hash>` first
<bonda_000>
oh you mean I need to clone that repo first?
<clever>
RPF releases new files several times a month
<clever>
and sometimes the new file may have useful data
<clever>
so every now and then i load up the latest one, and decompile it some more
<bonda_000>
okay that makes sense
<bonda_000>
but see there is a thing in all these bootloaders
<bonda_000>
you only seem to be able to access ISP regs at 2nd or 3rd bootloader stage otherwise they are unreadable
<bonda_000>
its weird
<clever>
probably power management stuff
<clever>
oh, one problem with the unstripped binary
<clever>
every string is tagged with the $S symbol
<clever>
ghidra is convinced thats code
<clever>
select the whole range (click first, shift click last), hit c to clear the type, then right click the first, data, string
<clever>
and for the unstripped binary, at offset ef2ed68, you have the `isp_module` symbol, change the type (t) to `void*[97]` and it will make far more sense
<clever>
that is sort of a vtable, for the isp class
<bonda_000>
I have it in my decompile as well
<bonda_000>
they do have something like a function table for each module and that's where these strings come I suppose
<bonda_000>
like if I scroll all the way up in the main decompilation file the first thing in there is a pointer PTR_PTR_s__CAMERA_POOL_END_0ef3a280 its at code address 0ec02800
<bonda_000>
thats the beginning of my decompiled .text section
<bonda_000>
and then I search for the start_x entry point within the bootcode (my start_x entry address is cec00200) so I search for a scalar 3468689920 and it's not in the bootcode so it must be constructed somewhere in the code
<clever>
bonda_000: bootcode.bin understands elf files, and will respect the entry-point defined in the elf headers
<clever>
that is very handy when running custom .elf files
<clever>
let me load up bootcode-vc4-2022-11-04-f3db2058e9d1f0cefb4c1aff5dbd9fd2d2749233.bin...
<clever>
there it is, at offset 800065d0, is a call to what i named fixup_apply, that takes fixup.dat (a binary patch), and directly applies it over the whole damn start.elf file, headers and all
<clever>
then it uses memcpy to copy a chunk of code to 0x60008000, search for that!, this is the stub, that changes L2 cache settings, and then jumps to start.elf
<clever>
line 6 applies the fixup.dat patch, based on the desired gpu_mem, and where the top of ram actually is
<clever>
something from 9-14 parses the new elf file (i figured this out on a different .bin version)
<clever>
16 copies the stub to some special sram
<clever>
i think 18-21 is an L2 cache flush?
<bonda_000>
well part of that is an armstub which is arm code
<clever>
and i think 22, is where it goes off to the stub in sram, and uvar8 is the elf entry-point, which came from 10
<clever>
the armstub is in start.elf
<clever>
bootcode.bin is pure VPU assembly
<clever>
bootcode.bin's only job, is to bring the lpddr2 controller online, then load start.elf into dram, and execute it
<clever>
start.elf's job, is to boot the rest of the system, drop an armstub at 0, turn on the arm core, and then provide runtime services (like the ISP) to the arm
<bonda_000>
yeah mine didn't decompile to the point of recognizing memcpy
<clever>
FUN_800002e8 from line 22, is just a `j r1`, its very obvious that the pastebin half was c, and FUN_800002e8 is asm in a second file
<clever>
yeah, memcpy isnt labeled at all in the .bin files
<clever>
you just have to find it manually
<clever>
i forget how, but ghidra can generate a database of function patterns and names
<bonda_000>
so that last link you sent is your bootcode entry code?
<clever>
and then auto-apply that to future decompiles
<clever>
the last pastebin is the end of bootcode.bin, right as it passes control off to start.elf
<clever>
and yes, it is huge, my .git is 24gig in size
<bonda_000>
so thats the firmware you are decompiling right?
<clever>
one of them, yes
<clever>
that link takes you directly to the unstripped version
<bonda_000>
oh damn thats 10 year old firmware
<clever>
yeah, thats how long ago they messed up and leaked all the symbols
<clever>
as far as i know, they never repeated that mistake
<clever>
but it still supported the isp and camera stuff, so the answers should be in there
<bonda_000>
have you generated the pattern folders for your vc4 processor or is it what you also have to do manually?
<clever>
when i import a file into ghidra, i rename it manually
<bonda_000>
oh yeah it is unstripped
<bonda_000>
all the paths of source files are there
<clever>
and names for every symbol
<clever>
and if we knew how the debug data worked, it may even be possible to recover the source
<bonda_000>
ok but then why the bootcode you were telling me about you picked a 2022 one
<bonda_000>
and not the same one from that 10 year old unstripped drop. I'm trying to follow what you do
<bonda_000>
none of the bins I fed ghidra it recognized, so I had to put mine into .elfs manually alas start_x.elf
f_ has joined ##raspberrypi-internals
<bonda_000>
the only problem I see is that this unstripped code is for VideoCore III
<bonda_000>
isn't it?
<bonda_000>
10 years ago is before RPi3 came out
<bonda_000>
that's why I'm thinking populating some patterns using this unstripped version and trying to apply it to the one that I know works on my 'puter
<clever>
oops, i keep getting distrcted with virtio code elsewhere
<clever>
bonda_000: the pi0-pi3 are all virtually identical internally
<clever>
the only real difference is which arm core is jammed on the side, and the VPU boot rom
<clever>
so the isp, hvs, v3d, and all that stuff, performs the same way on everything in that range
<clever>
the only major difference, is that the bcm2835 clocked some things at 250mhz, but the bcm2836(pi2) and newer, used a different process node, and clock at 500mhz
<clever>
bonda_000: as for why i picked a recent bootcode.bin, i just picked a random one in ghidra, to use as an example
<clever>
i named the hash, because i was initially using offsets into the binary, which are only valid for that version
<clever>
but once i found 0x60008000, i realized you can search for that, and find the same code in any binary
dolphinana has joined ##raspberrypi-internals
<bonda_000>
i guess i will just stick to wrapping the .bin into elfs since its pretty much the same thing
<bonda_000>
i'm just new to the ghidra interface and getting a bit lost in the features atm
<clever>
yeah, i was planning on moving to elf only as well
<clever>
and i did struggle with ghidra too, but after using it for several years, its all easy now
<bonda_000>
and these excerpts that you shared with git gists
<bonda_000>
they kind of resemble what I see in my bootcode except for the functions that you manually named I guess
<clever>
yep
<bonda_000>
at the end you have
<bonda_000>
do {blink_failure4();}while(true);
<bonda_000>
mine says
<clever>
yep, thats what causes the bootcode.bin to blink the led 4 times, when start.elf cant be found
<clever>
then the bootcode.bin, has to go thru the entire start(_x).elf file, and add 768mb to every address
<bonda_000>
to EVERY address?
<clever>
yes
<clever>
everything that is pointing within the elf
<clever>
but not MMIO
<bonda_000>
I mean
<bonda_000>
gpu_mem=30 means first 30 MB of RAM are used by VC4
<clever>
nope, LAST 30mb
<clever>
thats the reason its such a mess
<clever>
everything from 0 to (1024-30) belongs to the arm
<clever>
and everything from (1024-30) to 1024, belongs to the gpu
<bonda_000>
its a bit confusing how on Broadcom data sheet SDRAM is split in four slice one slice per 'alias region'
<bonda_000>
what from arm is just straightforward 0x0 - 0x3fffffff
<clever>
that is just repeating the same 1gig of ram 4 times
<bonda_000>
but thats the awkward VC4 memory map is why its sliced like that
<clever>
the top 2 bits of the address, are cache control flags
<clever>
any read from 0 to 0x3fffffff, is allowed to access both the L1 and L2 cache
<clever>
any read from 0x4000_0000 to 0x7fff_ffff is allowed to reach from the L2 cache, but a cache miss wont load the data into the L2 cache
<clever>
anything from 0x8000_0000 to 0xbfff_ffff will use the L2 cache normally, but ignore the L1 cache
<clever>
and anything from 0xc000_0000 and up, ignores both L1 and L2
<clever>
the reason for all of that, is that you can write to something in the "0 alias" (L1/L2 work normally), then you can do dma from the "4 alias" (L2 reads, but doesnt allocate)
<clever>
now the dma will respect what is in the L2 cache, so you dont have to flush the cache
<clever>
but the dma wont trash your L2, by filling it with data
<clever>
originally, the VPU ran the entire system, and the videocore was arm-less
<bonda_000>
thats what I want to do run it arm-less
<bonda_000>
so that the 4 cores dont waste power
<clever>
you may want to look at my lk-overlay stuff then
<bonda_000>
so I'm with you on the split but my VC4 should have all 1GB
<bonda_000>
brb
<bonda_000>
isn't dma only supposed to work on RAM
<clever>
and all 4 of those aliases, are the same 1gig of ram
<bonda_000>
and L1 and L2 caches are filled as the processor accesses memory
<clever>
the problem, is when the cpu writes something, and the data only lives in the cache
<clever>
you want dma to read the most recent version of the data
<bonda_000>
yeah thats what write through is for
<clever>
you can either do a costly cache flush, or you can just give dma access to the cache
<clever>
on x86, the pcie controller is part of the cpu, and it has permission to snoop on the caches, so you never have to flush
<clever>
on the vpu, the dma can access the L2 cache, but not L1, and you use the top 2 bits of the address, to control what dma will do
<clever>
but dma cant access the arm L1/L2 cache at all (design flaw by broadcom)
<clever>
so the arm must flush caches all the time
<bonda_000>
so on VC4 if the dma sees the L1/2 data and RAM data differ its going to take the cache version of it?
<clever>
the L2 cache is just another slave on the main bus
<clever>
so any access to the 0/4/8 alias, will just route the request to the L2 cache
<clever>
if the L2 cache has the data, thats a cache hit, it returns the data, and jobs done
<clever>
if the L2 cache doesnt have the data, it will then set bits 30/31 to 1 (moving it to the c alias), and send the request back out onto the bus, where dram will respond
<clever>
if the original request was to the 4 alias, then the L2 cache doesnt cache the response, and only forwards it on
<clever>
but if the request was for the 0 or 8 alias, the L2 cache will fetch an entire cache line, and cache it
<clever>
while also forwarding the answer to the original request
<clever>
this is the source that my open firmware uses
<bonda_000>
well then I think what I'll try to do now is try to put the regdump at the exit point of the bootcode and see it if hasn't enabled ISP by that time
<bonda_000>
so after it finished writing start_x.elf to SDRAM and before it branches there
<bonda_000>
at bootcode stage everything is in 128KB L2 Cache
<clever>
the only job bootcode.bin has, is to bring the lpddr2 controller online
<clever>
so its not going to be turning the ISP on at any point
<bonda_000>
so when the system is not yet booted you cant see these registers, as well as when its fully booted you can't see the ISP registers also
<clever>
you need to ask the firmware to turn on the ISP first
<bonda_000>
idk I tried playing with PWRMAN registers IMAGE register and it didn't go well
<bonda_000>
instead of flipping the bit on ISPRST it killed HDMI
<clever>
ive had trouble getting hdmi to even start working
<clever>
bonda_000: i went over the entire function, and labeled every MMIO address
<clever>
you can click `code` to see the non-diff version
<bonda_000>
I have references to DAT_7e100108 and how do you give it a name?
<clever>
2024-04-28 09:15:46 < clever> so, for PM_IMAGE, first i go into ghidra, in the memory window, and i create a PM block at 0x7e100000, 0x1000 long, volatile
<clever>
re-read the ~5 messages below that one
<clever>
then repeat that dance, on every address you see in the decompile
<bonda_000>
yeah so it doesnt replace it to PM_Image in the code but still useful
<clever>
adding it to the memory window, just changes it to read_volatioe()
<clever>
2024-04-28 09:16:05 < clever> then i can use `g` to goto 0x7e100108
<clever>
2024-04-28 09:16:13 < clever> and `l` to label it as `PM_IMAGE`
<clever>
2024-04-28 09:16:23 < clever> then `t` to set the type to `uint`
<bonda_000>
ah
<clever>
this then creates a symbol, and it becomes read_volatile(PM_IMAGE)
<bonda_000>
forgot the t
<clever>
reading the decompile, now that its done, i can see that if param2 is 2, it will do someting in CM_PERIICTL, PM_IMAGE, and VEC_DAC_MISC, so thats likely turning on the VEC (composite video encoder)
<clever>
4, it will do some things with H264
<clever>
8, the ISP
<clever>
0x10, i'm not sure
<clever>
0x40, the v3d, i think
<clever>
0x2000, the CCP2TX (one of the csi things)
<clever>
0x8000, DSI1
<clever>
0x10000, usb
<clever>
0x400, hdmi
<bonda_000>
uint or uint32_t* ?
<clever>
just `uint` for MMIO
<clever>
its not a pointer
<bonda_000>
there's like 5 different uints
<clever>
just hit enter, and it usually picks the right one
<clever>
another issue with the unstripped elf, every string has a $S symbol on it, which makes them show up weirdly
<bonda_000>
it didnt become read_volatile(PM_IMAGE) for me but rather PM_IMAGE = 0x5a001000
<clever>
i see a ldconfig_get_config($S) in the decompile
<clever>
screenshot?
<clever>
if i double-click on the $S, i can then use `c` and right click->data->string, to fix the types up
<clever>
and it becomes ldconfig_get_config("config_hdmi_boost")
<bonda_000>
no I mean the code using this address turned from Ram7e100108 = 0x5a001000 is now PM_IMAGE = 0x5a001000
<clever>
sounds like you forgot to check volatile in the memory window
<clever>
line 89, the isr (interrupt service routine), it will read the status reg, write it back to status, stash that in a struct, and signal completion of irq_sem
<bonda_000>
yeah that repo is not much different from what we have aside from more register definitions, it still talks to the VC4 through mmal-vchiq
<clever>
line 212, the mmap function for this char device, it will map RHEA_ISP_BASE_PERIPHERAL_ADDRESS when you try to mmap the char device
<clever>
think of this like mmap on /dev/ttyS0
<clever>
the driver is free to map whatever it wants
<bonda_000>
well, I guess if you could enable ISP on RPi then that driver would work on RPi too
<clever>
but that driver is only 2% of the code
<clever>
they hid the real logic in userland
<clever>
likely a closed-source library
<clever>
and because its not linking with linux, it doesnt have to respect the GPL and reveal the source
<clever>
loop-holes
<clever>
the only thing the driver does, is manage the clock, the irq, and expose things via mmap
<clever>
you can now clearly see, that isp_init creates the event flags, the thread, and then gets the event flags
<clever>
isp_exit will delete the event flags
<clever>
and isp_task, will set the flags (the very same flag init gets), so that means isp_init wont finish, until the isp_task() thread has reached this point
<clever>
and the do/while loop, then gets flags
<clever>
and the int handler sets the flags
<clever>
so basically, the int handler will unblock the thread, and then the thread takes action
<bonda_000>
i see that it takes some local_34 variable and compares it with bit fields
<clever>
exactly the same thing linux was doing with its complete(&dev->irq_sem);
<bonda_000>
thats the flags variable
<clever>
and it gave the addr of local_34 to flags_get
<clever>
so local_34 likely contains the flags the irq handler signaled
<clever>
and then based on those flags, isp_task() does different things
<bonda_000>
isp_task = &gp->isp_task;
<bonda_000>
so gp here is some global struct that holds RTOS tasks
<bonda_000>
their handles and stats and so on
<clever>
gp is basically just .data
<clever>
it holds everything
<clever>
so, as an example, if the flags contains 0x40, it calls isp_sw_stage_handler
<clever>
but isp_int_handler never sets that bit directly
<clever>
0x30...
<clever>
not found either
<bonda_000>
line 100 void isp_task(void)
<clever>
this may take days to unravel, but the header you found is massively valuable
<bonda_000>
line 120 isp_task = &gp->isp_task;
<bonda_000>
isnt it a name conflict
<clever>
it is
<clever>
ghidra doesnt care
<clever>
but if you tried to paste this into a file and compile, it will fail
<bonda_000>
undefined1 *isp_task has the same name as the function name and its all pointers
<clever>
i dont always check for collisions when naming things
<bonda_000>
yeah i guess so
<clever>
ghidra even lets you put invalid characters in a symbol name
<bonda_000>
but what does that mean though
<bonda_000>
you didnt name it like that manually did you?
<clever>
i did name everything in that line
<bonda_000>
you wrote isp_task = &gp->isp_task;?
<clever>
i used `l` to rename the variables
<clever>
and i created a gp_area struct
<clever>
as i explained above
<clever>
2024-04-28 11:44:20 < clever> down in the data type manager, under your elf file, create a new struct, call it gp_area, and make the size at least 0x2000
<bonda_000>
so what was it before
<clever>
re-read everything starting here
<bonda_000>
yeah i remember the gp part
<clever>
it was just `gp_uaff + 0x1234`
<clever>
creating the struct, and changing the type, turned it into `local_gp->field_0x1234`
<clever>
then i renamed everything, to make more sense
<clever>
you want a single `struct gp_area` for the entire binary
<clever>
and just change every `unaff_gp` into a `gp_area*`
<bonda_000>
ah I still have these dollar signs and something still isn't decompiling correctly
<clever>
as i said, $S is a string constant
<clever>
use `c` to clear that, then right click the first character, data, string
<clever>
the official compiler puts a $S on every single string, and ghidra blindly assumes symbols in .text are functions
<clever>
the rpi3-bootcode project, is more of just a demo of a basic bootcode.bin, which you could then expand upon
<clever>
vc4-stage1 is a bootcode.bin that functions similarly to the official one, it brings dram online, then loads lk.elf, from ext4!! (it lacks fat support, lol)
<clever>
the rpi3-start project, builds a start.elf, which is compatible with the whole pi0-pi3 range
<dolphinana>
(I use vc4-toolchain which I built by myself)
<clever>
you can either use the closed bootcode.bin to load it
<clever>
or you can use the open vc4-stage1 to load it
<bonda_000_>
okay its unpacking 1 channels
<bonda_000_>
taking its time
<clever>
it will be a bit slow the first time, since it has to build gcc, but its entirely automated, so you will have far fewer issues
<bonda_000_>
okay its done
<bonda_000_>
the nix part
<clever>
now try `nix-shell` in the lk-overlay directory
<bonda_000_>
yeah its like hung
<bonda_000_>
no prompt
<clever>
its probably doing network and cpu stuff
<clever>
check `top` in another window
<bonda_000_>
yeah its busy
<bonda_000_>
using up CPU
<clever>
sounds good, just wait
<bonda_000_>
its installing a lot of things
<clever>
yep
<bonda_000_>
is that all dependencies for your OS?
<clever>
most of them are just for building gcc
<bonda_000_>
i see
<clever>
i need to get some sleep, but once its done it will drop you back into a shell, that says nix-shell, like i showed above
<clever>
just run make as i showed above, and youll get a .bin out
<clever>
just copy that bin to an sd card, and rename it to bootcode.bin
<bonda_000_>
alright
<bonda_000_>
have a good night
<bonda_000_>
it failed though
<bonda_000_>
says I need a x64 linux
<bonda_000_>
I'm gonna get that going just in case we need it tomorrow
f_ has quit [Ping timeout: 260 seconds]
f_ has joined ##raspberrypi-internals
f_ has quit [Remote host closed the connection]
f_[xmpp] is now known as f_[not-xmpp]
f_[not-xmpp] is now known as f_[xmpp]
Ad0 has quit [Ping timeout: 268 seconds]
Ad0 has joined ##raspberrypi-internals
jcea has joined ##raspberrypi-internals
<bonda_000_>
I keep getting the error: a 'x86_64-linux' with features{} is required to build 'nix/store/ijvhxlzfpigvmgzg3bmg9a11wc7kp106-source.drv', but I am a 'aarch-linux' with features {benchmark, big-parallel, kvm, nixos-test, uid-range}
<bonda_000_>
I have windows on my main machine so I guess I will have to do it manually
<clever>
bonda_000_: yeah, there is some issues in the nix code, and it can only build the cross-compiler on x86
<clever>
bonda_000_: the nix page i linked, includes directions for WSL2, which is windows based
<clever>
so you can try things there
<bonda_000_>
can't I just build binutils and then build lk?
<bonda_000_>
like manually
<clever>
bonda_000_: yeah, you can still try that route as well
<bonda_000_>
i don't see any windows instructions only the bash script
<clever>
bonda_000_: WSL2 lets you get a full linux while on windows
<clever>
or you can just install ubuntu under virtualbox
<dolphinana>
hii, I'll go sleep now
<dolphinana>
good night ^^
<bonda_000_>
good night
<dolphinana>
and happy hacking! ^^
<bonda_000_>
wait I only can have wsl1 with my windows version
<bonda_000_>
i may just get a partition on a hard drive for that