15:02 <linkmauve> Besides the proprietary rknpu2 and rknn-toolkit{,2}, has there been any reverse engineering done on the NPU side?

15:17 <linkmauve> If not I think I’m going to start exploring that area.

17:34 <linkmauve> phh, is that your doing? https://github.com/phhusson/rknpu-reverse-engineering

17:36 <phh> yes

17:37 <phh> linkmauve: don't be dumb like me, and look at the TRM which completely describe it before reversing it

17:37 <phh> (reversing it did help me understand how it worked)

17:42 <phh> to help understanding it, one thing to note is that it is NOT a CPU. it doesn't have an instruction set. As a first abstraction, consider it's just memory-to-memory hardware blocks. you set their config, the memory pointers, set their START register, and wait for them to finish their task

17:42 <linkmauve> Unrelated, but the GPU can use YUV formats as both input and output?!

17:42 <linkmauve> (I’m reading the TRM and “input” contains “npu”… ^^')

17:43 <linkmauve> phh, makes sense.

17:44 <phh> it has a more advanced mode where it can chain commands (actually it's one hardware component of the NPU that simply writes the registers you asked it to, kinda like a DMA key-value writer), but I don't think it's very useful [1], and it'll just clog your brain

17:45 <phh> [1] it basically just gets you to be able to ignore most IRQs. I doubt you'll get micro-second NN tasks which would make this useful

17:45 <phh> either way, once you've written the ""basic"" mode, the advanced mode is a simple evolution of the basic one

17:46 <linkmauve> Is it described in another document than this one? I couldn’t find useful mentions of it in there: https://opensource.rock-chips.com/images/2/26/Rockchip_RK3568_TRM_Part1_V1.3-20220930P.PDF

17:46 <phh> uh, actually I've had much better understanding of the NPU than what I wrote...

17:47 <linkmauve> Heh. :)

17:48 <phh> linkmauve: look at RKNN section of the RK3588 documentation

17:49 <phh> kinda have to hope that the change isn't too big between those two...

17:49 <phh> i seem to remember that rk3588 had SRAM while rk3568 didn't?

17:50 <phh> linkmauve: you have full documentation for rk3568 in part 2 https://dl.radxa.com/rock3/docs/hw/datasheet/Rockchip%20RK3568%20TRM%20Part2%20V1.1-20210301.pdf

17:50 <linkmauve> Thanks!

17:58 <phh> (rk3588 has three cores, that actually means three identical IPs at various memory offsets, but that have a flag to compute things together, but let's just forget it)

18:01 <linkmauve> The RDMA thingy (is that the advanced mode you mentioned?) reminds me of the display list feature of old OpenGL.