<ALTracer>
My friend asked me which program displays FFT over streamed/captured waveforms, and I recommended ngscopeclient to him
<ALTracer>
Basically he wants to add STM32H7 ADC-backed online telemetry to his firmware and push samples over USB FS CDC-ACM
<ALTracer>
But after spending some hours building scope-hal he replies that Windows UART stuff is not implemented
<azonenberg>
That would be easy to add
<azonenberg>
We just never had demand for it, or anyone who had the time to test
<azonenberg>
Would probably be an hour or two of coding for a windows dev (which I'm not)
<azonenberg>
pretty sure there's been an open github ticket for a couple of years and nobody cared enough to spend time on it lol
<ALTracer>
I see the feature as an easy add, because I inadvertently tested/debugged BMDA and bmpflash backends for effectively winser.sys into blackpill-f411ce libopencm3 cdc-acm
<azonenberg>
Yeah i dont think it would be hard at all. lib/xptools/UART.cpp/h is the main thing you'd need to work on
<azonenberg>
that may in fact be the only thing you have to change
<ALTracer>
azonenberg: if there is a pending GH PR, that makes it not starting from scratch, I'd consider completing it either myself or with friend
<azonenberg>
since it looks like SCPIUARTTransport just calls out to the UART wrapper class and relies on it to handle all the platform specific stuff
<azonenberg>
I think there is a feature *request*
<azonenberg>
but no code
<azonenberg>
and actually no the ticket never got filed
<azonenberg>
i was confusing it with the one for USBTMC which is also not supported on windows right now
<ALTracer>
Second question is which protocol would he speak from his CubeFW_H7 C firmware? OLS? I know Pulseview has no FFT, so we're not aiming at sigrok support... but it would be a plus
<azonenberg>
this is the only bit yo'd have to change
<azonenberg>
wait what sigrok has no fft support? lol
<azonenberg>
i knew we were ahead of them in feature set but didnt think it was THAT bad
<ALTracer>
Oh, huh, USBTMC would be the next best thing I'd recommend -- we use a Hameg HMO1002 at $dayjob and it has TMC, VCP, and Ethernet
<azonenberg>
Most of the gear our serious folks use has ethernet
<azonenberg>
So socket stuff is supported widely across all of our major platforms
<azonenberg>
GPIB, USBTMC, and UART i think are all only on linux right now
<ALTracer>
well maybe I didn't dig deep enough into sigrok-cli. Edge Pulseview did not have a button for it, only LAs and binary decoder tables.
<ALTracer>
Maybe the new multimeter desktop client may have it
<azonenberg>
Anyway, as far as protocol goes, regardless of transport (usbtmc, uart, or TCP socket) you will probably want some variant of SCPI
<azonenberg>
but despite the S in "SCPI" standing for "standard"
<azonenberg>
there's a huge amount of variation between implementations and really no two are the same in terms of command set lol
<ALTracer>
Migrating from non-composite CDC-ACM to full lwip Ethernet NCM/ECM is a nontrivial effort for this one feature, and Win7 only speaks RNDIS not ECM (ask me how I know)
<azonenberg>
lol yeah understood
<azonenberg>
if you dont need a lot of data transfer bandwidth adding windows support to the uart transport class is by far the lowest friction route
<ALTracer>
Maybe not GPIB. Either ASCII-clear ols-compatible or VXI (?)
<azonenberg>
then just run SCPI data over the serial link
<ALTracer>
Bandwidth is USB FS, he pushed 1 MiB/s over MSC gadget, it won't get faster than that
<azonenberg>
ah cool yeah thats fine if you arent doing large waveforms
<azonenberg>
You can look at programming reference guides for any random oscilloscope to get a feel for how the commands typically work
<azonenberg>
the actual integration would probably be making a new oscilloscope driver class and exposing whatever diagnostics are on the firmware as scope channels
<ALTracer>
I personally used some rp2040 firmwares including sigrok-pico, and they are better&worse than a fx2la I bought from WeAct, so
<azonenberg>
basically you derive from SCPIOscilloscope and implement all of the virtual methods needed, stubbing out ones that don't make sense (e.g. setting offset on a channel that doesn't have hardware offset capability)
<azonenberg>
We really need to write up a developer guide at some point (/me pokes balrog to get doxygen built)
<azonenberg>
we have decent end user docs but the dev-facing docs are not great
<ALTracer>
Wait, so is it a "oscilloscope" or a "multimeter"/datalogger at that point? Like, the desktop client can ask to start printing capture and stop, but the fw is running regardless of that and measuring stuff always
<ALTracer>
It's a CNC controller
<azonenberg>
So first off you can implement multiple device classes in a single device
<azonenberg>
as a lot of especially lower end gear might be a scope, meter, and func gen for example
<azonenberg>
Generally speaking semantically, an oscilloscope does triggered acquisition of a block of samples
<azonenberg>
while a multimeter outputs a single reading of each channel any time you query it
<azonenberg>
(a meter that supports high speed datalogging would likely expose that feature in our API via the scope device class)
<ALTracer>
er, there he goes on GH
<ALTracer>
Caveat that we're both native Russian speakers, and I speak English better than him; so I hang in IRC/Discord more. He might not.
<ALTracer>
I doubt scopehal project would be interested in that one-off student project. So we basically patch the sources and build for msys2 ucrt64 then carry the binary around?
<azonenberg>
We support plugins, so you could write the driver as a plugin for exactly this reason
<azonenberg>
One thing to be aware of is that, similar to the Linux kernel, we don't currently guarantee any level of ABI stability (the project is still rapidly evolving and code is changing often). So you cannot practically distribute binary plugins
<azonenberg>
i.e. you have to distribute the plugin as source and compile it against a specific version of ngscopeclient/libscopehal
<azonenberg>
and that plugin binary has to be paired with that version of scopehal and if you update scopehal you have to recompile the plugin
<azonenberg>
But it eliminates the need to maintain a fork of scopehal with patches to add a driver
<azonenberg>
drivers, transports, and filter/protocol decode/math blocks are all dynamically created and plugin friendly
<ALTracer>
Like LGPL Qt5 redist as DLLs? So ngscopeclient.exe itself is upstream of a release tag version, but we just add device support code in a branch and build the DLL against that?
<azonenberg>
So this isnt a licensing issue its a practical issue. We are BSD licensed which is permissive and allows closed source plugins etc
<ALTracer>
or fetching scope-hal headers of an API version is sufficient
<azonenberg>
it's just that if scopehal changes and your plugin doesn't, memory layouts of classes etc might have changed
<azonenberg>
and your binary might no longer work on the new version
<ALTracer>
I see how this is a C++ blessing and a problem
<azonenberg>
(there is no release tag version of ngscopeclient yet, we're aiming for a v0.1 release this summer)
<ALTracer>
Yeah, no licensing questions (yet)
<azonenberg>
we expect for v1.0 to start providing some level of stability so that binary plugins will work with the same major version of scopehal into the future
<azonenberg>
But there is no timeline for when that will happen
<azonenberg>
for v0.x series, plugin binaries have to be built against a specific commit of scopehal and be recompiled if you update
<ALTracer>
specific commit? Don't you do v0.1.x monthly tags?
<d1b2>
<johnsel> Nope, not yet
<azonenberg>
No, it's just been rolling "everyone runs git master or close to it" so far. We're hoping to have our first formal v0.1 release in a few months
<ALTracer>
Ah, beta. Well, we can make it work as long as it's not seriously broken
<azonenberg>
trying to polish up documentation and cross platform stuff more before then
<azonenberg>
Once 0.1 is out i expect to have more frequent 0.x releases tagged
<ALTracer>
Sounds like BMD git HEAD but no snapshots, I see
<azonenberg>
But can't promise any specific frequency. We're a small team and none of us are doing this full time or close to it
<azonenberg>
We have a few part time corporate supporters but not the level of funding/backing we'd need to get a couple of full time devs (yet)
<azonenberg>
But the level of interest is slowly growing, you're actually the third student team i know of working on a project that plans to integrate with scopehal
<azonenberg>
there may be more who haven't reached out to us but i know of at least three including you
<azonenberg>
And at least one startup instrument vendor is looking at using ngscopeclient as their official UI
<ALTracer>
azonenberg: UART.cpp is pretty small, Read/Write and Connect/Close. Also linux-y with termios. I think the Windows impl will have all four methods fully different, speaking from blackmagic experience
<ALTracer>
BSD means I can't copy GPL code from BMDA or bmpflash?
<ALTracer>
qt5serialport makes it all go away, but you're not using Qt at all
<d1b2>
<johnsel> you can simply do CreateFile / ReadFile / WriteFile under windows
<d1b2>
<johnsel> maybe use SetCommState
<ALTracer>
there are timeout and buffering shenanigans unfortunately
<ALTracer>
There are technically two backends in BMDA, libusb one and tty serial one for when none of libusb/hidapi/libftdi1 were available on spartan buildsystems
<ALTracer>
bmpflash also had a libusb win backend, which required Zadig to bind
<ALTracer>
so they ditched it and rewrote to winser constraints, bumping perf and CPU usage in the process
<lethalbit>
but I filed the issue where I thought was approps due to the last non stdlib frame in the backtrace
<d1b2>
<johnsel> #16 LogSink::WrapString (this=this@entry=0x5eba28f61b40, str="Warning: Socket read failed (errno=14, Bad address)\n") at ~/scopehal-apps/lib/log/log.cpp:91
<d1b2>
<johnsel> #5 0x0000715ed7ab52a7 in malloc_printerr (str=str@entry=0x715ed7bbde78 "munmap_chunk(): invalid pointer") at malloc.c:5772
<d1b2>
<johnsel> weird error
<lethalbit>
Yeah, dunno *shrug*
<lethalbit>
might be UB somewhere else
<lethalbit>
might look into building things with asan/ubsan to see if I can catch it again
<d1b2>
<johnsel> I'm not sure if it's a cleanup error
<d1b2>
<johnsel> from a socket issue
<d1b2>
<johnsel> or if it's invalid memory access inside the socket read
<lethalbit>
yeah, dunno,
<lethalbit>
might be Rigiol specific because i've been having no end of issues trying to get it to even capture a waveform
<lethalbit>
i'll poke at it later:tm:
<azonenberg>
lethalbit: innteresting, let us know if you see anything. i have never seen that before
<azonenberg>
it's possible the rigol driver is corrupting memory and this is just a symptom
<azonenberg>
what i do know is that i've seen one or two cases with the Tek driver of the connection being lost, the driver aborting, and something being corrupted during cleanup when that happens causing a segfault inside the abort
<azonenberg>
This may or may not be the same bug
<azonenberg>
(crashes that happen after you've already aborted are fairly low priority for obvious reasons)
<lethalbit>
Yeah, i'll try to poke it some more and if anything new happens I'll let ya'll know
<lethalbit>
I think in general the Rigol driver needs some poking, or something becaue I can't even get it to trigger properly, or even single-shot, it /sometimes/ works, but thats less rare than it outright crashing
<ALTracer>
next time (build and?) link with -fsanitize=address for more details, I don't think UBSan will catch the double free. And valgrind is too perf-expensive
<azonenberg>
yeah we have an option in the cmake script to build under asan
<azonenberg>
SANITIZE i think
<azonenberg>
as far as the rigol stuff goes i recently did some work on it for juh's scope and it was working well for her at the time
<lethalbit>
the asan/ubsan combo has been pretty good for the software I work on, it does catch some silly ub mistakes that sometimes happen, so yeah i'll try that.
<azonenberg>
but i dont know which rigol you're using, she had a ds1000z i think?
<lethalbit>
I'm using a DS1054Z
<azonenberg>
(also i dont think we've done much testing of scopehal with ubsan but we should... i do use asan regularly to track down crashes)
<ALTracer>
Also Gentoo QA wiki taught me to `thread apply all bt full` to also see all the locals and other threads. Can you please extract it while you have the coredump?
<azonenberg>
ok yeah i think thats the same model she had i'm surprised you're having issues
<lethalbit>
yeah I seem to be cursed like that :v
<ALTracer>
how recently? makes for a shorter git bisect
<lethalbit>
literally git HEAD
<ALTracer>
azonenberg: I meant juh's case
<lethalbit>
i've enabled sanitized, i'll rebuild and test once AUR updates are done rebuilding KiCad :v
<ALTracer>
kicad 8 hit aur? nice
<lethalbit>
I use dragnyans kicad-nighlty packages, so it's not /quite/ 8
<lethalbit>
for stable 8 I had to pull the arch pkgbuilds and update them myself to build stable 8 for local
<ALTracer>
looks like vowstar and dlan overtook maint of kicad in gentoo, thanks to them we have 8.0.0 since 3 days ago in ~amd64
<ALTracer>
or not, they're proxy maint
<ALTracer>
whatever, Arch is not my slice of pie
<lethalbit>
I'm happy ngscopeclient doesn't crash my GPU now at least :v
<lethalbit>
a few months back it would just outright lock it up
<ALTracer>
lethalbit: hm? which GPU vendor was affected? btw I only use Intel iGPU and Zen APU
<lethalbit>
My box has an AMD Vega 56,
<lethalbit>
It's stable now, but a while ago it was rough
<azonenberg>
yeah we fixed a couple of shader bugs over the past few months
<azonenberg>
ALTracer: it was months ago, thanksgiving time frame, i patched up some broken commands and it was working for her
<azonenberg>
but idk how aggressively she was using it
<lethalbit>
wee asan to the rescue I guess
<lethalbit>
`==750981==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x7a126bfbe891 at pc 0x7a12b266bfd8 bp 0x7a12681fea70 sp 0x7a12681fe218`
<lethalbit>
Unsure if it's directly related, seems like a different crash
<azonenberg>
So is the rigol driver overrunning the socket rx buffer?
<azonenberg>
it seems like maybe header_blocksize somehow is >250K
<azonenberg>
so there should be a bounds check there, but that should never happen and likely indicates the data coming back from the scope is complete garbage
<azonenberg>
i.e. it'll save you from memory corruption but not make the thing actually work
ALTracer has quit [Ping timeout: 264 seconds]
ALTracer has joined #scopehal
<d1b2>
<altracer> got disconnected from ISP side, does my Revolution IRC client also misbehave by not sending messages from offline/pauses?
<ALTracer>
I'm not sold on L840 doing a recv of arbitrary scanf'ed header_block size without buffer overrun checks. Needs its LogDebug on L822 uncommented.
<azonenberg>
yeah we absolutely need to bounds check there
<ALTracer>
anyway, I'll just resend manually.
<ALTracer>
500000 is exactly double your 250k limit coming from somewhere, maybe it's CH1/CH2?
<ALTracer>
can't say without hands on Rigols, I'm not that rich to own a $500 scope at home
<lethalbit>
mood, maybe one day i'll have a fancy Keysight or LeCroy scope, a girl can dream
<azonenberg>
lethalbit: anyway yeah looks like the L840 recv needs to boundscheck header_blocksize first
<azonenberg>
(but the bigger question is why the scope is sending that much data - or if the parsed value is even accurate)
<lethalbit>
dunno,
<lethalbit>
i'll keep poking at it
<ALTracer>
technically you can just double the buffer because desktop RAM is infinite
<azonenberg>
Lol
<lethalbit>
I've already had Yosys OOM me out of 128GiB RAM before, I don't need more of that plz
<ALTracer>
ofc that doesn't fly on blackpill-f411ce which has a grand total of 128 KiB SRAM, but then again it has no EMAC to do sockets
<ALTracer>
oh you're one of FPGA people with Vega and Threadrippers and obscene amounts of DDR4
<azonenberg>
This is a STM32H7
<lethalbit>
I'm just a simple catgirl,
<lethalbit>
nice nice
<azonenberg>
The current firmware is 67 kB of flash and 56 kB of global SRAM, plus 128 allocated to stack but i'm not using anywhere near that
<azonenberg>
there's a sshd on there too
<azonenberg>
no dynamic allocation whatsoever
<ALTracer>
I can boast with my STM32MP157 on Monday if you like :) H7 on steroids
<azonenberg>
MP is a cortex-A series though right?
<azonenberg>
i like working on M's. If I outgrow the H735 i'll probably go to i.MX RT
<azonenberg>
1 GHz M7 with three ethernet macs gogogo :D
<ALTracer>
yeah, I maintain devicetrees and patch OpenSTLinux for Cortex-A7 at $dayjob
<azonenberg>
It's a MCU that wants to be an AP when it grows up lol
<azonenberg>
anyway, my current project is using the H735 paired with a kintex7, the MAC is actually living on the FPGA
<ALTracer>
it also has a Cortex-M4 remoteproc in it, kinda like imx7/imx8
<azonenberg>
then i use the octospi on the MCU to access a FIFO in block RAM on the FPGA that captures inbound frames destined for the MCU
<azonenberg>
and i have a hook into my IP stack to do qspi commands to send/recv them
<azonenberg>
(right now lots of blocking IP but i'll make it interrupt driven / DMA eventually)
<ALTracer>
Can't wait for STM32MP253 to drop with dual GMAC and 64-bit and whatnot
<azonenberg>
Nice. I still like M series though lol
<azonenberg>
one of my big things is not running linux on things that don't *need* linux
<azonenberg>
forget throwing down ddr and spending a hour building a kernel image
<lethalbit>
found another overflow :v
<ALTracer>
you'd like RZ/A1L
<azonenberg>
my firmware would fit on a 5.25" floppy with room to spare
<azonenberg>
and my ram usage would fit on my old smith corona pwp-80
<lethalbit>
also it turns out if you click the arm button twice accidentally it'll blow up
<lethalbit>
:v
<azonenberg>
Sounds like the driver is buggier than i thought lol
<lethalbit>
yeah, tracking things down
<ALTracer>
So much for feature parity with Pulseview :P
<azonenberg>
(let's just say the lecroy driver won't do that)
<azonenberg>
ALTracer: it's very driver specific. the application core and filter graph stuff is pretty good
<lethalbit>
still can't manage it to get any triggers still either
<azonenberg>
but drivers are maintained by people who have those scopes
<azonenberg>
i dont have a rigol
<ALTracer>
that locks up and crashes when I hit Run twice in a row with fx2lafw
<azonenberg>
So code quality varies widely on them
<azonenberg>
lolol
<ALTracer>
you people have triggers?
<lethalbit>
`Header block size = 101110111011`
<lethalbit>
lmao
<azonenberg>
lethalbit: yeah it sounds like your scope's firmware is hosed and is sending gibberish back to you (and/or the driver desynced from the scope somehow)
<lethalbit>
didn't crash tho!
<lethalbit>
it eventually recovered
<azonenberg>
and the driver is insufficiently robust to fail gracefully vs segfault when that happens
<azonenberg>
lol
<azonenberg>
(this is par for the course for these cheap scopes i see firmware instability on them all the time
<lethalbit>
I got a whole 3 waveforms out of it
<azonenberg>
otoh i've also got a tek mso6 to hang more times than i can count
<azonenberg>
lol
<azonenberg>
now you see why i like lecroy :p
<azonenberg>
They're not cheap but you do get what you pay for
<lethalbit>
yeah,
<lethalbit>
shame it's eternally out of reach T~T
<azonenberg>
old ones are coming down in price
<ALTracer>
is that L822 %zu? does not sound like protocol response, rather comtrade-like encoded data
<azonenberg>
i see windows xp based lecroys going for 2-3k on ebay now
<lethalbit>
i'm currently a NEET so anything above 50 bucks is out of my range :v
<azonenberg>
ok fair enough lol
<ALTracer>
okay maybe I'll try the hameg hmo1002 then with ngscopeclient in hopes it behaves better and triggers (I don't think it will stream)
<azonenberg>
I dont think we have any drivers for that yet but feel free to write one
<ALTracer>
as I said 2 hours ago it speaks VXI and TMC
<azonenberg>
pretty much all of these scpi protocols are a nightmare to recover if they desync
<azonenberg>
they're line oriented with in-band binary data blobs bracketed by ascii headers
<ALTracer>
I know a bit of COMTRADE
<azonenberg>
What I use for all of my own hardware, and anything i write a socket server for (like pico)
<azonenberg>
is line oriented scpi for control plane
<azonenberg>
and a dedicated socket with binary framing for waveform data
<azonenberg>
keep control and data plane separate
<ALTracer>
huh, ASCII .cfg and binary FLOAT32 .data are separate files for reasons
<ALTracer>
well they used to store ASCII .dat as printable text, but it had terrible entropy
<ALTracer>
line oriented... BMPremote is like and not like this. It has in-band ASCII commands, some of them (responses likewise) come with a hex-encoded payload at 50% efficiency
<ALTracer>
Because F1 and F4 can't really afford that many dedicated VCPs, because endpoints...
<ALTracer>
I followed ST on swapping RNDIS with ECM in MP15, checked both in USBOTG over HS and FS, it's 200Mbps vs 8
<ALTracer>
Maybe a humble F4 really should rather do ECM over 3 EPs, and not dual ACM plus binary Trace Capture for TraceSWO
<ALTracer>
azonenberg: aren't you that Xilinx+KSZ based switch designer?
<_whitenotifier-7>
[scopehal] lethalbit opened pull request #863: Fixed crashes due to `header_size` and `header_blocksize` occasionally being much larger than associated buffers. - https://github.com/ngscopeclient/scopehal/pull/863
<azonenberg>
ALTracer: I'm making a switch but it's not KSZ based
<azonenberg>
There's one KSZ9031 on the management console but i'm using VSC8512s for the edge ports and SFP+ for the uplink
<azonenberg>
Using KSZs for a 24/48 port switch would be ridiculous, that many RGMII lanes across a 19" chassis would be a nightmare to route
<ALTracer>
alright, I remembered
<azonenberg>
much better to use GTXes and QSGMII and have a single transceiver channel run four switch ports
<azonenberg>
thats what i have the sshd and scpi stack running on
<ALTracer>
you support Rohde&Schwartz, right? RTC1002 looks suspiciously like HMO1002
<azonenberg>
We have two R&S drivers
<azonenberg>
rs supports the RTM3000 and possibly others of that generation and their ancestors
<azonenberg>
rs.rto6 supports the RTO6, RTP, and other stuff on the new software platform
<azonenberg>
i cannot speak to any of the older R&S models, i dont think anyone has ever tested what works with it. We'd like to hear feedback, good or bad, so we can at least have an idea of what's out there
<azonenberg>
R&S bought Hameg some years back so it's very possible that newer R&S low end scopes are based on the Hameg software stack
<ALTracer>
H735, Spansion FL... I'd happily run NuttX w/eth on it, nuttx-apps included. Maybe they have a sshd.
<azonenberg>
I do not know how much commonality that has with R&S scopes
<azonenberg>
The spansion flash is for the FPGA not the MCU
<azonenberg>
the MCU is fully self contained at the moment, just quad spi to the FPGA. but no external memory
<ALTracer>
what do you suggest me building it on? Windows 7, 10, or Linux Mint 19, 21? I have a quad-boot
<azonenberg>
Debian based Linux distros (i think mint is?) are the most well tested, reliable platform right now since I primarily develop on debian
<ALTracer>
Mint is Ubuntu LTS with extra changes by Clement Lefebvre and the team
<azonenberg>
and new features with platform-specific stuff tend to hit linux first for the same reason
<azonenberg>
Yep. and ubuntu is debian based
<ALTracer>
as you wish :)
<azonenberg>
So i expect it to work well although i am not aware of anyone who has specifically used it on mint
<ALTracer>
Are you aware of anyone using scopehal on Gentoo? This one cat seemingly uses Arch
<ALTracer>
about time I merged world
<azonenberg>
Not off the top of my head. I know of people using hat-based distros, debian based, and arch
<_whitenotifier-5>
[scopehal] azonenberg closed pull request #863: Fixed crashes due to `header_size` and `header_blocksize` occasionally being much larger than associated buffers. - https://github.com/ngscopeclient/scopehal/pull/863
<_whitenotifier-5>
[scopehal] azonenberg eb96dc3 - Merge pull request #863 from lethalbit/master Fixed crashes due to `header_size` and `header_blocksize` occasionally being much larger than associated buffers.