f_ changed the topic of ##raspberrypi-internals to: The inner workings of the Raspberry Pi (Low level VPU/HW) -- for general queries please visit #raspberrypi -- open firmware: https://librerpi.github.io/ -- VC4 VPU Programmers Manual: https://github.com/hermanhermitage/videocoreiv/wiki -- chat logs: https://libera.irclog.whitequark.org/~h~raspberrypi-internals -- bridged to matrix and discord
bonda_000 has joined ##raspberrypi-internals
<bonda_000> this seems to load camera_func_table using dlshared_vll_load()
<bonda_000> undefined ** camera_driver_load(undefined4 param_1,undefined4 param_2,int *param_3)
<bonda_000> I'm not exactly sure where it is loading it from
<bonda_000> from some hidden place within the start_x.elf?
<bonda_000> it uses open() and read() assumes there is some sort of filesystem
<bonda_000> so these drivers must be somewhere within start_x.elf or fixup_x.dat
<bonda_000> all I can say so far is
<bonda_000> void * cdi_camera_open(int *param_1,int *param_2)
<bonda_000> calls open_camera_driver() which initializes separately camera_subsystem (CSI unicam block) and loads camera_driver by dloading some vll with camera_func_table
<bonda_000> this cdi_camera_open itself has input arguments and no xrefs so will have to look for that now
<bonda_000> it runs some internal filesystem probably derived from the .elf / fixup files
<clever> bonda_000: the fixup file is purely a binary patch, that modifies the elf to run from some other address
<clever> the pi can boot just fine without the fixup
<clever> but it will act like it has 256mb of ram
<bonda_000> any clue where those vll patches are?
<bonda_000> the symbol table is full of these
<clever> none currently
<clever> but i did find a big table, with names and function pointers
<bonda_000> camplus_open loads some vll symbol to get the function table but we can see it in the decompile
<clever> look at the xref's of isp_module, you should see a function
<clever> what was the function called? (i dont have ghidra open right no)
<bonda_000> absolutely zero xrefs
<bonda_000> when there should have been at least one (see right side)
<clever> then what is get_isp_module ?
<clever> change get_isp_module to return the isp_function_table struct from before
<clever> the fact that you tagged it as int, means its not an address, so it cant be an xref
<clever> ghidra is beliving that lie
<clever> ah yes, its gp based, thats why
<clever> setting gp doesnt create an xref
<clever> now search for the address of get_isp_module, youll find it in a big undefined1 array
<clever> how many bytes is that array? what is the symbol?
<bonda_000> it looks a bit nicer now but still no xrefs
<clever> yeah, gp doesnt make an xref, even when you set gp like you discovered
<bonda_000> 388 bytes
<clever> wrong direction
<clever> you want to search for ed0fdd8
<clever> 2024-05-09 22:52:48 < clever> now search for the address of get_isp_module, youll find it in a big undefined1 array
<bonda_000> _dummy_functab?
<clever> thats the one
<clever> how big is it?
<bonda_000> undefined1[1320]
<clever> and 1320/4 is?
<bonda_000> 430
<clever> now make _dummy_functab a `void*[330]`
<bonda_000> oh no
<bonda_000> 330
<clever> ah, didnt see the typo, i did the same math on my end
<clever> then search for ed0fdd8 again, and youll find its element [164]
<clever> and there is a "isp.vll" and "get_isp_func_table" right before it
<clever> and camplus_open called dlshared_get_vll_symbol(handle,"get_isp_func_table");
<bonda_000> yeah I see
<clever> and on closer inspection, _dummy_func is a repeating array of 3 pointers, `struct { char*vll; char*symbol; void*function;}`
<clever> and it has 110 of those
<clever> thats your missing xref
<clever> i think at one time, isp.vll was a file on the disk
<clever> and it only got loaded into ram when camplus_open calls for it
<bonda_000> now its just a part of .elf?
<clever> but the rpi firmware isnt allowed disk access, after linux has booted
<clever> so the _dummy_func table redirects the whole vll subsystem, to something already in the elf
<bonda_000> I thought it was filled with new pointers from the file we read
<clever> there is also a camplus.vll with a get_camplus_func_table symbol
<bonda_000> and get_camera_func_table is gonna be one of the two supported ones at that moment in time
<bonda_000> toshh2c or ov5647
<clever> and bingo, camera_info_parameter_get will try to load camplus.vll and call get_camplus_func_table
<clever> and that perfectly leads over to mmal_component_create_camera_info!
<bonda_000> camera_info_parameter_get as a function?
<clever> yeah
<bonda_000> I don't know if you read everything
<bonda_000> I wrote
<bonda_000> take a look at start_camplus()
<clever> which part of it?
<bonda_000> its same as the camplus_ block was passing around the isp_func_table within a struct
<clever> let me finish tagging this function first...
<bonda_000> start_camplus'es param_1 carries the camplus_func_table within a struct
<bonda_000> so its like one level above camplus
<bonda_000> it seems to be important, yet it doesn't seem to be a part of any particular block
<bonda_000> it goes together with try_ that is also not a member of any particular block, but has "cameraRIL" debug messages
<bonda_000> brb
<bonda_000> (**(code **)(*(int *)(param_1 + 0x1b6c) + offset) that's the camplus_func_table
<bonda_000> when it does *
<bonda_000> so is camera_info that huge struct that's being passed around?
<clever> cant make much sense of start_camplus yet
<clever> but mmal_component_create_camera_info is huge, because that looks like what mmal exposes to userland
<clever> if you then look at the symbol tree, under mmal_component_create
<bonda_000> start_camplus does to camplus_module exactly the same what camplus_module did to isp_module
<clever> youll find aggregator, camra_info, copy, miracast, null_sink, passthru, scheduler, splitter, video_convert
<clever> all of the things the video pipeline does
<clever> have you seen the old openmax docs?
<bonda_000> I just looked at some openmax stuff today
<clever> let me find the link...
<bonda_000> ?
<clever> yep, thats the one
<bonda_000> it says
<clever> and here, you can see a list of components
<bonda_000> don't use any of this with IL
<clever> over the openmax api, you can create an instance of each thing in here
<clever> and you can take any numbered port of the same color, and create a connection
<clever> so if you wanted to record the camera, you would create an instance of camera and video_encode, and then link port 70 of the camera to port 200 of encode
<clever> and then route encode's port 201 back to linux
<clever> and boom, you get an h264 stream
<bonda_000> it says
<bonda_000> dont use OMX with IL
<bonda_000> and IL is all over the vpu code that we were looking at
<clever> the whole omx interface has been replaced by mmal, and then replaced again by v4l
<bonda_000> IL or RIL I know that IL stands for interface layer
<clever> mmal was meant to be a simpler interface
<clever> but v4l is a standardized interface, that works outside of the rpi ecosystem
<clever> but if you then look at the camera component, youll see it has a bunch of properties on each port, like OMX_IndexConfigCameraISPOutputPoolHeight
<bonda_000> well
<bonda_000> the whole things seems to be built to the OpenMAX standard
<bonda_000> and that has three layers
<bonda_000> AL IL HL
<bonda_000> but mmal I thought was "access layer" but OpenMAX talks about "abstraction layer"
<bonda_000> maybe they are the same thing
<clever> bonda_000: https://picamera.readthedocs.io/en/latest/fov.html#division-of-labor look at the diagram in step 9
<clever> this seems to say that mmal runs over the vchi protocol
<clever> and i believe vcgencmd, openmax, and the broadcom gl, also run over vchi(q)
<clever> > Use MMAL instead of OpenMAX - it's simpler in the RPC calls, and in the client interface.
<clever> > They both go over VCHIQ, it's just the state machine that has to be maintained on the client side is simpler with MMAL.
<clever> yeah, and this post says that both omx and mmal go over vchiq
<clever> this is where i was saving those notes
<clever> and checking mmal_component_supplier_register, its very clearly just a singly linked list
<bonda_000> but if we are on VPU side we can ditch all the access layer
<clever> each entry has a next pointer, a pointer to the mmal_component_create_camera_info function, and a char[0x20] for a name
<clever> so when mmal asks for a certain component, you can walk this singly linked list, and then call mmal_component_create_camera_info
<bonda_000> what does this function look to you like
<bonda_000> void * try_update_and_start_capture(int param_1)
<clever> and bingo, mmal_component_supplier_create, will walk that list
<clever> it looks like try_update_and_start_capture just starts the capture on a camera
<bonda_000> is it calling the unlabeled (void *)FUN_0ee2cf76(); and (void *)FUN_0ee2cf76();?
<bonda_000> those seem to be hickups because of the switch statement within that function
<clever> that is just a label within the function for me
<clever> i think you need to recreate the function, because the $c's broke it
<bonda_000> pcVar8 = *(code **)(*(int *)(param_1 + 0x1b6c) + 0x40);
<bonda_000> remember how the isp_func_table was an offset within that huge 0x70000 byte struct?
<clever> yeah
<bonda_000> this is the same, but the camplus_func_table is at offset 0x1b6c
<bonda_000> wasn't too difficult to deduce after you showed me the trick
<clever> aha, found some more stuff
<bonda_000> so this one is calling camplus_get_ctrl entry number 16
<clever> the miracast code, is calling back into the mmal core, to create a miracast_camera, ril.video_encode and miracast_frame_ctl
<clever> so one larger component, is then reusing other smaller components
<clever> and there, i found the top
<clever> a simply worker() function, that calls vchiu_queue_pop, and has a switch-case on it
<clever> that then calls into the various things
<bonda_000> I don't think I can understand what mmal is doing until I figure out where the level below it, ril, resides
<bonda_000> for me, it's mostly working my way to the top to figure out the contents of that param_1 that always seems to be a large control structure
<clever> starting at worker() looks like a great place to me
<bonda_000> a worker() if I remember correctly is what reads the messages from Linux side
<clever> yep
<bonda_000> but aren't we already relieved of that if we have pure VPU at our hands
<clever> but starting at the worker, will show you how linux gets to the various things
<bonda_000> so we can get rid of that top level messaging system
<clever> let me cross-reference to linux now...
<bonda_000> no, I do look up the camera driver to get hints where to look at
<clever> [clever@amd-nixos:~/apps/rpi/linux-pi5/drivers/staging/vc04_services/bcm2835-isp]$ ls
<clever> so if i start here, i think
<clever> in bcm2835-v4l2-isp.c
<clever> we have a vchiq_mmal_component_init(dev->mmal_instance, "ril.isp",
<bonda_000> on VPU side all it's gonna do is call a few RTOS functions and return I'm pretty sure
<clever> then in vchiq-mmal/mmal-vchiq.c
<clever> it calls create_component
<clever> m.h.type = MMAL_MSG_TYPE_COMPONENT_CREATE;
<clever> which sets the message type like so
<clever> and in mmal-msg.h, we can see that MMAL_MSG_TYPE_COMPONENT_CREATE is 4
<clever> and there is a `case 4` in worker(), tht does mmal_server_do_component_create()!
<bonda_000> piVar3 = (int *)rtos_calloc_priority(sVar2 + 0x215,4,1,0xedb4ad0); is the gist of the mmal_component_create_core on the VPU side
<clever> so now we can seamlessly trace the references in linux, all the way into the mmal core
<clever> and then keep tracing them down into camplus
<bonda_000> my bad it's alway truncating my functions because of mem_ ops
<bonda_000> I do see set and get in the mmal_init_port function
<bonda_000> void mmal_component_init_control_port(int *param_1)
<bonda_000> that's what mmal_component_create_core is calling
<bonda_000> it writes 4 function pointers
<bonda_000> into the param_1 struct
<bonda_000> mmal_component_enable_control_port; mmal_component_disable_control_port; mmal_component_parameter_set; mmal_component_parameter_get;
<bonda_000> enable and disable do nothing, just return
<bonda_000> yeah the only good thing is that mmal functions and input parameters are probably documented somewhere since it has almost one-to-one syntax on the Linux side
<bonda_000> only with a vchiq_ prefix on the Linux side
<clever> yep
<bonda_000> so that means send a vchiq request to call the similar function on the vpu side
<clever> so you can now trace things backwards to linux and other mmal clients (libcamera, vlc), and see what they mean
<clever> yep
<bonda_000> once I get ril
<bonda_000> no
<bonda_000> If mmal uses ril
<bonda_000> I need to know what arguments it passes to it
<clever> aha, param_2 from mmal_server_do_component_create, is a `struct mmal_msg` from linux
bonda_000 has quit [Read error: Connection reset by peer]
bonda_000 has joined ##raspberrypi-internals
<clever> aha, param_2 from mmal_server_do_component_create, is a `struct mmal_msg` from linux
bonda_000 has quit [Quit: Leaving]
fury999io has quit [Ping timeout: 264 seconds]
fury999io has joined ##raspberrypi-internals
f_ has joined ##raspberrypi-internals
bonda_000 has joined ##raspberrypi-internals
<bonda_000> Does vcos_sprintf need all this at the input?
<bonda_000> void vcos_snprintf(char *param_1,size_t param_2,char *param_3,undefined4 param_4,undefined4 param_5,
<bonda_000> undefined4 param_6)
<bonda_000> it calls
<bonda_000> int vsnprintf(char *__s,size_t __maxlen,char *__format,__gnuc_va_list __arg)
<bonda_000> I need to fix this I think
<bonda_000> int _doprint(undefined *param_1,undefined4 param_2,int *param_3,int **param_4,undefined4 param_5,
<bonda_000> undefined4 param_6,undefined param_7 [172])
<bonda_000> this is still a part of _doprint and converting those byte to code did not make any more sense
<bonda_000> I think this one just needs varargs
<bonda_000> :clever
<bonda_000> but it's still weird these functions start_camplus and try_ are not a part of any particular block
<bonda_000> every block seems to have all it's functions listed in some function table
<bonda_000> except this one
<bonda_000> my only suspicion is that it may be internal functions of the cdi block
<bonda_000> which is also camera related, I haven't found what cdi stands for
<bonda_000> Camera Device Interface it could be
<bonda_000> this looks like the top function of the Interface Laye
<bonda_000> r
<bonda_000> uint * camera_ril_create(uint **param_1,void *param_2)
<bonda_000> allocates memory at line 21 puVar3 = (uint *)rtos_calloc_priority(0x408c,4,1,0);
<bonda_000> sets up 4 ports
<bonda_000> writes a lot of magic values
<bonda_000> then actually writes to the param_1 pointer *param_1 = puVar3 + 0xdce;
<bonda_000> creates a "ca_process_thread"
<bonda_000> iVar4 = vcos_thread_create_classic
<bonda_000> ((int)(puVar3 + 1),"ILCamera",ca_process_thread,puVar3,
<bonda_000> (int)(puVar3 + 0x73),6000,10);
<clever> bonda_000: its a varargs function, so it only consomes what the format string needed
<bonda_000> Was thinking today of doing a VGA
<bonda_000> but I only have one DAC at the moment
<bonda_000> maybe I could drive it at triple the pixel clock and do switched capacitors thing?
<clever> bonda_000: https://www.pishop.ca/product/gert-vga-666-hardware-vga-for-raspberry-pi/ but you lose access to the uart
<bonda_000> yeah I've already seen it
<bonda_000> its pricey too
<bonda_000> takes 8 weeks to get it here
<clever> its just a bunch of resistors
<clever> you can find the schematic online and just make it on a breadboard
<bonda_000> I have a ton of capacitors, some relatively high-speed BJTs
<bonda_000> you know how these switched power supplies work
<clever> somewhat
<clever> the bcm2835 even has one right in the SoC
<bonda_000> it's off half the time but the charge is stored on the capacitor, the oscillator is picked so that it flip the switch back on before the capacitor discharges way too much
<clever> VDD_OUT is a 5v PWM signal
<clever> L5 and the capacitors smooth it out
<clever> VDD_FB is the feedback
<bonda_000> yeah so by the same principle
<clever> from a single register write, you can change the target that it tries to keep FB at
<bonda_000> if I need three analog signals and have only one dac
<clever> and the input of VDD_BAT, implies this SoC was designed for battery use at one time
<clever> why convert lipo->5v->1.1v when you can do lipo->1.1v direct
<clever> i think composite video would be simpler then vga
<clever> its a single signal, using a dac that already exists in the chip
<bonda_000> a dac is in the bcm2835?
<clever> yep
<bonda_000> never knew of it
<bonda_000> I know there's an adc within the tempctrl
<clever> that dac is how it generates pal and ntsc video
<bonda_000> I see
<bonda_000> well my current options are a VGA and HDMI
<clever> with the proper adapter cable, you can get video and audio out of the phone port
<clever> and if shipping is too bad, just solder a wire to the right pin of the port
<clever> i have yet to get hdmi working on the open firmware
<bonda_000> yeah I remember these from PlayStation 1 days
<bonda_000> I have an EXT RGB in my flat screen TV
<bonda_000> has a lot of pins
<bonda_000> but sadly none of these three jacks PAL/NTSC connectors
<clever> that sounds like scart
<bonda_000> yeah 20 pin rgb
<bonda_000> ext rgb
<bonda_000> he is doing RGB444
<bonda_000> is that DAC within the BCM2835 documented anywhere?
<clever> sort of
<clever> these are my own docs on it
<clever> and the 1st one links to the linux source, where i got most of the info
<clever> internally, it can run in 3, 2, or 1 channel mode
<clever> 3 channel mode can produce rgb or component
<clever> 2 channel mode can produce s-video
<clever> 1 channel mode can do composite
<clever> but only 1 dac has ever been wired to an external pin
<bonda_000> what pin is that
<bonda_000> I don't see anything on the datasheet
<clever> which datasheet?
<bonda_000> bcm2835 the table with alt functions
<clever> its not on the gpio array
<clever> there is a dedicated pin just for the dac
<bonda_000> 46 through 53 are internal
<clever> its on pin P14
<clever> bottom right corner of page 1
<bonda_000> R15
<bonda_000> dac2b5?
<bonda_000> 2v5*?
<clever> page 2
<clever> dac_2v5, is just a 2.5v supply for the dac
<bonda_000> VID_DAC
<bonda_000> P14
<bonda_000> found it
f_[xmpp] has quit [Changing host]
f_[xmpp] has joined ##raspberrypi-internals
f_ has joined ##raspberrypi-internals
f_ has quit [Changing host]
<bonda_000> so if only one has a wire then its only a 1 channel mode?
<clever> yeah
<clever> and because only 1 channel mode works, they never released source for configuring the other modes
<clever> so all we have to go on, is a mode enum that leaked in a header
<bonda_000> I think switching R,G,B at triple speed with capacitors isn't a bad idea and hooking it up to the VGA
<bonda_000> so if pixel clock is 30MHz, say
<bonda_000> a pixel comes in, software put red component on the dac, flip the switch
<clever> but the hardware isnt able to interleave the subpixels like that
<bonda_000> now the capacitor is charged to the red component
<bonda_000> right
<bonda_000> can do it is software
<clever> timing would be far more unstable
<clever> and it would use a lot of cpu
<bonda_000> its like a demux in a sense
<bonda_000> 1 to 3 demux
<clever> yep
<bonda_000> my prof was laughing in University at this term
<clever> gotta run now, lunch time!
bonda_000 has quit [Ping timeout: 246 seconds]
bonda_000 has joined ##raspberrypi-internals
<bonda_000> uint rtos_malloc_priority(uint param_1,uint param_2,uint param_3,uint param_4)
<bonda_000> Do you also have this function with 4 params?
<bonda_000> trying to figure out what camera ril is allocating with this call
<bonda_000> puVar3 = (uint *)rtos_calloc_priority(0x408c,4,1,0);
<bonda_000> beacuse this is later on is passed downward to camplus_block and somehow becomes 0x70000 byte struct
dolphinana has joined ##raspberrypi-internals
f_ has quit [Quit: To contact me, send a memo using MemoServ, PM f_[xmpp], or send an email. See https://vitali64.duckdns.org/.]
arsenijs has quit [Quit: https://quassel-irc.org - Chat comfortably. Anywhere.]
arsenijs has joined ##raspberrypi-internals
<bonda_000> This is where both cores are trapped in the beginning
<bonda_000> also the boot sequency is tricky. I had to alter the Ghidra a little bit. It sneaks the address where a piece of code is in the midst of data into the lr and it looks like a simple "return" statement
<bonda_000> so from vccrypt_start it doesn't go back to main, but goes to one of the two FUN's instead, depending on which core I am
<bonda_000> it was cluttered in that area for me and it didn't label that as a function, I had to do that manually
<bonda_000> this is the full sequence for the primary core
<bonda_000> the secondary core goes to the last piece of it, _os_main_hook which starts the rtos on the secondary core and also ends up in an infinite loop
<bonda_000> is there a way to redirect VPU log messages to the uart?
<clever> bonda_000: uart_2ndstage=1 in config.txt should do that
<dolphinana> hi o/
<bonda_000> hello
<bonda_000> is it going to hurt the VPU if there are no device tree files?
<bonda_000> it's where the hardware is described isn't it
<clever> bonda_000: the firmware knows where all of the hardware is and doesnt need dtb
<clever> the dtb is only there to help linux understand things
<bonda_000> ok
<bonda_000> I'll just give it a try see if I can get any messages
<bonda_000> is it a 115200 baud?
<clever> yes
bonda_000 has quit [Read error: Connection reset by peer]
bonda_000 has joined ##raspberrypi-internals
<bonda_000> but now since I'm going to the second stage my tftp folder should also contain config.txt
<clever> yep
dolphinana has quit [Quit: Leaving]
<bonda_000> <clever> add program_usb_boot_mode=1 to config.txt, reboot, then check vcgen again
<bonda_000> <...>
<bonda_000> <clever> it just enables booting from usb when SD cant be found
<bonda_000> so where is going to get the config.txt with this param if the SD isn't inserted
<bonda_000> or were we just checking if my board is capable of USB/Ethernet booting?
<bonda_000> that's when you asked me to tell you my OTP value
<bonda_000> <clever> bonda_000_: if you boot normal raspi-os and run `vcgencmd otp_dump | grep 17:`, what does it report?
<bonda_000> <bonda_000_> 1020000a
<bonda_000> <clever> usb is not enabled
<bonda_000> <clever> add program_usb_boot_mode=1 to config.txt, reboot, then check vcgen again
<bonda_000> I was booting without config.txt when I tried lk
<bonda_000> just fine