lanefu changed the topic of #armbian-rockchip to: Armbian - Linux for ARM development boards | Rockchip SoC | www.armbian.com | This channel is relayed to the equivalent Discord channel | this channel is logged
juri_ has quit [Ping timeout: 265 seconds]
juri_ has joined #armbian-rockchip
juri_ has quit [Read error: Connection reset by peer]
<DC-IRC> [Discord] <beohoff3174> Has anyone had any experience with using the NPU?
<DC-IRC> [Discord] <beohoff3174> I glanced at the driver situation and it looks a hot mess.
juri_ has joined #armbian-rockchip
<DC-IRC> [Discord] <menu> it *is* a hot mess!
<DC-IRC> [Discord] <doofin000> Hi guys, I'm going to buy a RK3588 SBC, which brand do you recommend?
<DC-IRC> [Discord] <menu> whichever meets your needs
<DC-IRC> [Discord] <doofin000> orange pi vs radxa? which have better docs and cares about community?
<DC-IRC> [Discord] <beohoff3174> I think Radxa has a better community, plus the Collabora folks are using those devices to test their drivers
<DC-IRC> [Discord] <beohoff3174> but it shouldn't take you that long to validate the community yourself
<DC-IRC> [Discord] <menu> both are pretty terrible, neither cares about the end-user
<DC-IRC> [Discord] <Werner> opi is worse rumors say 🤷‍♂️
<DC-IRC> [Discord] <infinity_q> radxa is better, opi is worse, but the community has stemmed up around the lack of support
<DC-IRC> [Discord] <infinity_q> radxa is better, opi is worse, but the opi community has stemmed up around the lack of support
<DC-IRC> [Discord] <zeeto9> I have some experience. Depends what you are trying to do. Currently I have 3 ip streams being ingested into an opi5+ frigate utilizing the npu for object detection on all of them. Uses roughly 10% cpu.
<DC-IRC> [Discord] <beohoff3174> @zeeto9 , I was thinking about Speech to Text streaming and potentially text detection/recognition. Are you using the vendor driver?
<DC-IRC> [Discord] <zeeto9> Yea I’m using rknpu, I think 0.97. I have not attempted any AI for text or speech recognition. Simple voice commands I would like to add one day for easy on/off/control. I’ve even thought about using an esp32-s3 for speech commands, but haven’t tried it due to not wanting to over complicate it either.
<DC-IRC> [Discord] <beohoff3174> Are you using a custom model?
<DC-IRC> [Discord] <zeeto9> At the moment I am using frigates-nas model, small since it was tested with their frigate:stable-rk image and surprisingly works very well for what it is. Custom models are up next since I want to utilize frigate as more than just “nvr” and also have it be a central repo for my videos to run object recognition, which is nice since it organizes it all in a sql db behind the sc <clipped message>
<DC-IRC> [Discord] <zeeto9> enes. In general, as long as the model is valid and functioning, pointing to config towards to custom .rknn is all that should be required to change
<DC-IRC> [Discord] <zeeto9> At the moment I am using frigates-nas model, small since it was tested with their frigate:stable-rk image and surprisingly works very well for what it is. Custom models are up next since I want to utilize frigate as more than just “nvr” and also have it be a central repo for my videos to run object recognition, which is nice since it organizes it all in a sql db behind the sc <clipped message>
<DC-IRC> [Discord] <zeeto9> enes. In general, as long as the model is valid and functioning, pointing the config towards the custom .rknn is all that should be required to change
<DC-IRC> [Discord] <zeeto9> Model average speed is ~30ms
<DC-IRC> [Discord] <zeeto9> Currently set to identify 12 objects
<DC-IRC> [Discord] <beohoff3174> ok good to know
<DC-IRC> [Discord] <beohoff3174> I think for now I might push text to speech onto the cpu cores
<DC-IRC> [Discord] <beohoff3174> I was thinking about potentially dropping some scene detection change into like a vulkan shader, but I'd have to kick the tires on how good the vulkan support is on Panthor
<DC-IRC> [Discord] <beohoff3174> maybe I'll try the text recognition/detection with the npu
<DC-IRC> [Discord] <beohoff3174> just realized my OPi5+ doesn't have sound/microphone so might reflash this bad boy
<DC-IRC> [Discord] <beohoff3174> especially since I lost ethernet too for some reason
<DC-IRC> [Discord] <beohoff3174> I was playing with BitNet last night: https://github.com/microsoft/BitNet
<DC-IRC> [Discord] <beohoff3174> because you can run llama3.2 on like 2-3 gb of ram. Not sure what you can do with it though, the model is pretty brain dead from my casual play with it
<DC-IRC> [Discord] <zeeto9> You might be able to take a model like: vosk, deepspeech, or wav2letter in onnx or tensflow and convert to rknn.
<DC-IRC> [Discord] <beohoff3174> The model zoo for rockchip actually has whisper in it
<DC-IRC> [Discord] <zeeto9> Ohh! I might give that a try this week. If it’s simple enough, I might try deploying it to npu core1 and see how multi core with different models works
<DC-IRC> [Discord] <beohoff3174> I've got some code that is more server level (with cuda) using one of the libraries that leverages whisper, but I want to see how realtime and edge you can get
<DC-IRC> [Discord] <beohoff3174> https://github.com/airockchip/rknn_model_zoo
<DC-IRC> [Discord] <beohoff3174> would love to hear your experiences with it
<DC-IRC> [Discord] <zeeto9> I suppose if I really want to find out how well it works, I could also attempt to setup an esp32-s3 with wake words to see how well it’s “real-time” compares. The esp32 is only .5 tops, but it specifically has ai speech acceleration for this usage. Could be interesting.
<DC-IRC> [Discord] <beohoff3174> The home assistant guys are using that chip right? I think they were publishing custom software for it
<DC-IRC> [Discord] <beohoff3174> I bought a device to try and keep pace with what they were doing, but just ran out of time
<DC-IRC> [Discord] <zeeto9> The esp32-s3 is being used in a lot of very low powered AI sensors. The .5 tops can handle up to a few frames a second for object recognition. The idea is that the simple npu on device does “detection” for what’s interesting and if it’s interesting, sends it to edge server for full frame processing. Sort of like how frigate handles detection vs identification
<DC-IRC> [Discord] <beohoff3174> I'm surprised it can pull bandwidth of the image itself, but maybe I'm giving too much emphasis on how big video is... let alone the model
<DC-IRC> [Discord] <beohoff3174> most of my experience with esp's are the 8 bit versions though, using them for low powered devices, so I haven't looked at it as a development platform
<DC-IRC> [Discord] <beohoff3174> I'll expose my ignorance here, what signals are image sensors using? UART? SPI?
<DC-IRC> [Discord] <beohoff3174> Usb 1.1 with a maximum speed of 12Mbps, not bad
<DC-IRC> [Discord] <beohoff3174> I think YUV422 at 1920 x 1080 was around 7 megabyte, so you could potentially do 1 fps in just data transfer
<DC-IRC> [Discord] <beohoff3174> though most ML models do much smaller images, like 224 x 224 (and I forget what color space) so you could probably cut that down quite a bit
<DC-IRC> [Discord] <zeeto9> Oh boy, wav2vec2 is going to take an hour to download at its blazing fast 100kb/s. 😆😄
<DC-IRC> [Discord] <menu> depends on the sensor, there's a lot of MIPI CSI sensors
<DC-IRC> [Discord] <zeeto9> On the esp32-s3, I think it’s the DCI which is a combination of GPIO pins like data, sync, clock, etc. I never thought about it too much since the esp32-s3 has built in support for some cameras. Plenty of options that plug and play with the software. While I am terrible at C, esp-idf for what it is is a great sdk. I wish everything was documented and supported like esp32 and raspberry pi’s.
<DC-IRC> [Discord] <zeeto9> Though, I will give rockchip some credit specifically when it comes to their rknn toolkit2. That software and documentation is one of the few areas rockchip seems to provide thorough documentation and regularly keeps it updated. Everytime I look at the repo it’s slight more organized with more examples.
<DC-IRC> [Discord] <beohoff3174> I was surprised they just provide a binary for the userspace driver though
<DC-IRC> [Discord] <beohoff3174> or that's what I thought I saw
<DC-IRC> [Discord] <beohoff3174> I didn't dig much
<DC-IRC> [Discord] <runaway97> compare models not brands. yes, opi is one of the worst when it comes to software and attitude, but it's not like you're going to enjoy the official OS images of any of the sbc vendors. except maybe khadas and recently radxa who have started producing saner images. i would still go with either armbian or joshua riek ubuntu though.
<DC-IRC> [Discord] <runaway97> for example between rock 5b ane opi 5 plus i would get the opi 5 plus because you just get more for the buck (+cheaper) and it's a bit better designed IMO
<DC-IRC> [Discord] <zeeto9> My biggest issue is I can figure out how to get the boards to work for my needs in user space level development, but I don’t have the ability to handle low level kernel updates, security, or even knowing what are security holes in areas like uboot etc
<DC-IRC> [Discord] <zeeto9> It makes me hesitant to even build something for another person and expose it to the internet. I don’t know what I don’t know.
<DC-IRC> [Discord] <zeeto9> is armbian the closest to mainline kernel for these boards? Even though it doesn’t have full hardware support, I’d consider that for situations that don’t require full support and mainly needs cpu/ram for services,docker, microk8s
<DC-IRC> [Discord] <zeeto9> I definitely agree on this. Even between the orange pi 5 variants, the plus has superior IO. Not just more, but often better/newer standards than 5b/pro/max for pci, Ethernet, lanes, etc