<d1b2>
<ehntoo> I'm going to take a wild guess that AVX512F support is probably less beneficial than the loop parallelization and add an #ifndef __clang__ the calls to the AVX512-accelerated functions in ScopeSyncWizard.
<azonenberg>
ah interesting. i bet clang's openmp implementation doesnt understand calling functions with different ISAs
<azonenberg>
Yes
<azonenberg>
long term in ngscopeclient that's all going to go GPU anyway
<d1b2>
<ehntoo> nods
<azonenberg>
I'm moving away from openmp in general as it proved to be a pain
<azonenberg>
using application managed threading to the extent practical
<azonenberg>
but it's low priority and i'm in no rush to remove all of the OMP
<d1b2>
<david.rysk> OMP is less of a problem than FFTS
<azonenberg>
yes removal of FFTS is higher priority
<azonenberg>
i think that will be lain's next focus once we merge the queue manager
nelgau has quit []
<d1b2>
<david.rysk> Is there a good example of the correct use of vkFFT in the current scopehal codebase?
nelgau has joined #scopehal
<azonenberg>
the fft filter
<azonenberg>
We will be removing fft entirely from the demo scope i think. there's better options
<azonenberg>
it didnt even produce great looking waveforms
<azonenberg>
spectrogram is the main remaining user of it
<azonenberg>
that will get trickier because it does a lot of FFTs
<azonenberg>
so we will have to rewrite some of our wrapper code to allow many FFTs to be batched
<azonenberg>
(so far none of our filters do >1 FFT. de-embed does a FFT and IFFT but those are separate kernels and each one is done on a whole buffer)
<azonenberg>
we need to figure out if, for example, we can do multiple FFTs within a single VkBuffer at various offsets
<azonenberg>
or if we will have to make a separate VkBuffer for each region we want to FFT, etc
<azonenberg>
ehntoo: is this ready to merge now you think?
<azonenberg>
or are you still making changes?
<d1b2>
<ehntoo> yeah - as long as the scopehal header change looks unproblematic to you, I think it should be good to merge. I'm still running some build time experiments, but they're probably best to land separately anyway
<d1b2>
<ehntoo> "scopehal header change" being the change in avx_mathfun.h, that is.
<_whitenotifier-7>
[scopehal] ehntoo 26c3b13 - Add AVX2 feature attributes to avx_mathfun header This seems to be needed for clang to properly compile with the correct ABI. Without this change, clang requires the whole binary to be linked with -mavx to avoid error.
<_whitenotifier-7>
[scopehal] azonenberg 475fb18 - Merge pull request #708 from ehntoo/clang-x86-64-avx-fixes Fix compile issues on x86_64 clang
<_whitenotifier-7>
[scopehal-apps] ehntoo a3c42e2 - Add first draft of a macOS github action runner
<_whitenotifier-7>
[scopehal-apps] ehntoo 10afb7f - Avoid AVX512F + OpenMP functions on clang Clang seems to have issues with the combination of openmp and the different ABI required for AVX512F usage.
<_whitenotifier-7>
[scopehal-apps] azonenberg d2ce6dd - Merge pull request #509 from ehntoo/add-macos-ci-runner macOS GitHub actions CI runner
<_whitenotifier-7>
[scopehal-apps] azonenberg 7c453dd - Added new "debug" menu for showing ImGui and ImPlot demos. Demo windows now disappear correctly when closed.
<_whitenotifier-7>
[scopehal-apps] azonenberg 5f2ffff - Timebase properties: shortened title for better tabbing
<_whitenotifier-7>
[scopehal-apps] azonenberg 62969ee - WaveformGroup: fixed bug when tabbing out
<azonenberg>
So, here's food for thought WRT how to do cursor value readout
<azonenberg>
ehntoo, lain: you two have only *tested* on arm64 macos right?
<azonenberg>
the CI builds are x86-64 but nobody's actually ran the apps on that platform
<d1b2>
<Darius> I can test on OSX x86 if you want
<d1b2>
<Darius> I don't have a suitable 'scope though
<azonenberg>
you can use the demo scope (driver "demo", transport "null", put any non-empty string in the path)
<azonenberg>
it'll display some synthetic waveforms
<azonenberg>
exists for precisely this purpose
<azonenberg>
what GPU does your mac have?
<azonenberg>
i have no idea what vulkan support if any may or may not exist
<d1b2>
<Darius> unless you want to slum it with a TDF2024
<azonenberg>
anyway, it sounds like at this point apple is going all in on apple silicon, right? like, they're not making intel hardware anymore?
<d1b2>
<Darius> Radeon Pro 560X
<d1b2>
<Darius> yeah no more Intel hardware
<azonenberg>
if that's the case, supporting older x86 macs is not the highest priority. that said, it will be good to know if it already works
<azonenberg>
go ahead, try it and report back
<azonenberg>
we probably wont put a lot of work into it if it fails
<azonenberg>
but if it works that will be nice
<d1b2>
<Darius> d
<azonenberg>
ok so here's an idea
<azonenberg>
cursor values are displayed next to channel names
<azonenberg>
readout for *markers* is in some kind of table or something
<azonenberg>
or maybe a tooltip if you mouse over the marker
<azonenberg>
ehntoo: so it looks like right now the main thing keeping GTK in scopehal is that filters return a Gdk::Color in GetColor() right?
<azonenberg>
then we also have ExportWizard which temporarily we can refactor into scopeexports
<azonenberg>
which is a glscopeclient-only module
<d1b2>
<Mughees> Yes it does…
massi has joined #scopehal
<d1b2>
<miek> fwiw, here's what my vna does - it shows the values for the active trace all the time and then has an option to display a table with all values
<d1b2>
<ehntoo> Yes, I've only actually tested on an ARM Mac. I've got an x86 MacBook here as well, I'll give the new CI artifacts a try on it today.
<d1b2>
<ehntoo> Yep, I'm pretty sure the gdk::color is the only tie to gtk in scopehal. I can knock out a refactor to use HTML color strings instead tonight.
<d1b2>
<ehntoo> Results of my x86 macbook test: 1. CI artifacts aren't useful out-of-box due to dylib path issues. Maybe something to look into later, I built from source instead. 2. glscopeclient on master gives an (expected) opengl error when adding a scope, ngscopeclient on master segfaults 3. Merging latest master into lain's macos branch, glscopeclient works (see screenshot), though ngscopeclient still segfaults
<d1b2>
<david.rysk> On 1, ideally an .app would be packaged
<d1b2>
<david.rysk> On 3, the crashes I was getting were due to the LPF calling into ffts
<d1b2>
<david.rysk> Turning off the LPFs allowed it to work
<d1b2>
<ehntoo> for 1, ideally, yes. that's not something I have any experience with, though. for 3, I don't think so. this is a real scope, not the demo scope.
<d1b2>
<david.rysk> Ahhh.
<d1b2>
<ehntoo> also ffts should work on x86 macos
<d1b2>
<ehntoo> actually, looks like the demo scope does crash glscopeclient, so maybe ffts doesn't work.
<d1b2>
<ehntoo> ngscopeclient is segfaulting before it even seems to open a window
<d1b2>
<ehntoo> I'll troubleshoot later though, need to get to $job
<d1b2>
<ehntoo> I lied, I noticed an apple problem reporter window and couldn't help looking at the backtrace for the glscopeclient demo crash. Yep, FFTS is broken on macOS x86 too. Thread 36 Crashed: 0 libscopehal.dylib 0x110ee9fe0 ffts_execute + 0 (ffts.c:172)
<d1b2>
<ehntoo> and the ngscopeclient segfault backtrace
<d1b2>
<ehntoo> (with master of this morning merged in)
<azonenberg>
well, yet another reason to leave FFTS
<d1b2>
<ehntoo> the ngscopeclient crash might be a little more interesting. looks like a queue submission is failing?
<azonenberg>
ehntoo: I have seen this with older GPUs that provide only a single vulkan queue when we try to use >1
<d1b2>
<ehntoo> this is on lain's macos branch that I was pretty sure had the new queue management, though
<azonenberg>
lain's queue manager work is meant to solve this but i dont know how complete it is in her pushed branch
<azonenberg>
specifically in ngscopeclient
<d1b2>
<ehntoo> ah, gotcha
<azonenberg>
if it's not fully integrated and we're still using more queues than the hardware provides that would explain it
<azonenberg>
also possible we have a different bug. but don't spend any time on it at this point
<azonenberg>
we know we have to merge the queue manager, and we know we have to remove FFTS
<d1b2>
<ehntoo> 👍
<azonenberg>
those are definite blockers, once they're fixed we'll test again and see what shakes out
<d1b2>
<ehntoo> every macOS system will expose 4 all-purpose queues with the way MoltenVK is implemented at present. It sounds like that may change when macOS Ventura comes out and MoltenVK can start using Metal 3.
<d1b2>
<david.rysk> Does Metal 3 require newer hardware?
<d1b2>
<ehntoo> I don't believe so, it's just a new api revision. Though ventura will deprecate more x86 macs.
<azonenberg>
ehntoo: so even older intel based macs will have four metal queues?
<d1b2>
<ehntoo> correct.
<azonenberg>
well the bigger question is how many queues apple silicon will expose now / in the future
<azonenberg>
if there's a cap of 4 in metal they might not have built more in hardware
<azonenberg>
anyway, the other thing is
<azonenberg>
we are still likely going over the limit
<azonenberg>
because that's four separate queue families with one queue in each
<azonenberg>
before lain's fixes we try to allocate all compute/transfer queues from one family
<_whitenotifier-7>
[scopehal-apps] azonenberg 0554c95 - Experimental tweak to build on >= 1.3.229 Vulkan SDK. See #508.
<azonenberg>
ehntoo: was it you who tried and had trouble building on the 1.3.229 Vulkan SDK?
<d1b2>
<ehntoo> no, I think that was @david.rysk
<d1b2>
<david.rysk> Yeah it was me
<azonenberg>
ah ok. Well in that case @david.rysk: please test 0554c95 and let me know if it compiles on 1.3.229 or later SDKs as well as whatever you're currently using
<azonenberg>
there's now an ifdef check that should, fingers crossed, build on everything
<d1b2>
<ehntoo> I would bet we can probably reclaim sufficient space for now by cleaning the built artifacts from the vulkan sdk after install
<d1b2>
<johnsel> I'll take a look at it today, I was planning to spend some time on scopehal anyway
<d1b2>
<johnsel> I'll probably just insert my 'rdp and sleep' script and run wiztree or something to see what takes up most space
<d1b2>
<johnsel> It would have been nice to set up even a temporary self-hosted runner but Windows is the one prickly OS as far as the CI OS install goes with the weirdly Azure specific scripting
<d1b2>
<johnsel> the other approach might be that it seem the MSI build should just be a normal build + some extra steps
<d1b2>
<johnsel> instead we have 2 completely separate buildtrees that are mostly duplicates
<d1b2>
<ehntoo> doing the single build for the MSI would also speed up that runner significantly.
<d1b2>
<johnsel> worst case I do // build 1 // upload rm -rf build1 // start build 2
<d1b2>
<johnsel> but yes it would be a huge optimization target
<azonenberg>
I mean, that will need to be done whether we self host or not to ~double the build speed, right?
<d1b2>
<ehntoo> got out of a meeting and had dumped all my mental stack of useful work stuff, so I gave master ngscopeclient another try on x86 macOS for the heck of it. and it works? quite nicely, even. haven't had it crash again through a couple re-launches, so I'm not sure what was going on this morning. Maybe I needed to do a clean build of the tree or something.
<azonenberg>
btw, right now our intensity grading is logarithmic
<azonenberg>
at some point i want to add a selector for log or linear
<azonenberg>
i think a lot of other scopes do linear
<azonenberg>
We have more flexibility because we integrate in fp32 domain rather than int8 like a lot of the cheap scopes do
bvernoux has joined #scopehal
<d1b2>
<ehntoo> looking at this, I think there might be either an off-by-one error in the offset handling or some other scaling error in the unit handling for this rigol. this wave is perfectly centered around 0 on the scope's display, but is centered slightly below it in ngscopeclient.
<azonenberg>
Pleaase do investigate
<azonenberg>
i have not yet implemented cursor readout so it's tricky to do exact verification that values are what you want when/where you want them
<d1b2>
<ehntoo> will do, I'll make that first priority after work today.
<azonenberg>
also test in glscopeclient on another machine w/ latest scopehal
<_whitenotifier-7>
[starshipraider] azonenberg 1d06af4 - Updated AKL-AV1 to v0.6
massi has quit [Remote host closed the connection]
<d1b2>
<johnsel> yes azonenberg it's rather doing it now vs doing it later, but it should happen at some point regardless
<azonenberg>
Yeah
<azonenberg>
well sounds like it just got more important to do sooner rather than later :)
<d1b2>
<johnsel> I have a few of those ideas that maybe make sense to put in tickets, for this specific problem I actually think bvernoux has gained some useful knowledge between his different approaches to building so it may make sense to get his input and do it later
<azonenberg>
If you want to start a more formal discussion, go for that too
<azonenberg>
I just want to see progress being made somewhere by someone
<azonenberg>
Whatever you think will give the best results long term
<d1b2>
<johnsel> reason being I am not very familiar with the makepkg script that is currently being used
<d1b2>
<johnsel> and I think there is a way to do without, I think I saw bvernoux use
<d1b2>
<johnsel> but regardless I'll do something to optimize the build today
<azonenberg>
yeah even if it's just cleaning intermediate artifacts, i want to stop getting false positive CI spam if i didn't actually break the windows build
<d1b2>
<johnsel> sure, that makes perfect sense
bvernoux has quit [Quit: Leaving]
<d1b2>
<ehntoo> ok, checked out the rigol under glscopeclient and everything looks fine. I put it on the highest voltage range with a 50 ohm terminator so that noise would be ~1 LSB and correlated the voltages reported by the scope's tracking cursor with glscopeclient - everything looks good. Double-checked with ngscopeclient and it also seems fine, so I'm not sure what was going on this morning. 🤷♂️
<d1b2>
<ehntoo> interestingly, the waveform data coming out over SCPI seems to have more "noise" in it than on the scope display? Certain ADC count changes (those that only last one sample, I think?) don't show up on the scope's display, even zoomed into the shortest timebase and in dot display mode.
<azonenberg>
interesting
<azonenberg>
i wonder if they do filtering for display
<azonenberg>
LeCroy at least does flatness corrections in DSP
<azonenberg>
you actually get visual artifacts if you force 8 bit transfer mode when downloading waveforms from the scope
<azonenberg>
even if it's only an 8 bit ADC
<azonenberg>
because the flatness corrected data has >256 possible values
<azonenberg>
So that's now a preference setting, you can optimize for fast download using 8 bit transfer format or best visual waveform quality using 16 bit format
<azonenberg>
i expect we'll be adding more settings like that for other drivers in the future to do various tweaks
<azonenberg>
But essentially, with a LeCroy when you ask for samples you are not actually getting raw adc codes
<azonenberg>
vs with say a rigol, you probably are getting stuff straight off the adc
<d1b2>
<ehntoo> yeah, I would imagine so.
<azonenberg>
For our own future scopes, I am planning to stream raw ADC samples, keeping the scope-side processing as light as possible
<azonenberg>
then have a measured .s2p for each channel of the scope that you can download over the network interface when you connect
<azonenberg>
then apply flatness calibration clientside with the de-embed filter
<d1b2>
<ehntoo> this is the waveform that I've confirmed is "correct", btw. I think what I might be observing is just a partial-LSB internal DC offset in the scope resulting in a preference for +1 LSB results over -1LSB results.
<azonenberg>
incidentally, it is possible to calibrate that out with some effort
<azonenberg>
basically you can average a zillion samples then subtract the average
<azonenberg>
displacing the whole waveform by a fraction of a LSB
<azonenberg>
you will see this more noticeably if you're using a resolution enhancement filter to trade BW/sample rate for improved vertical resolution
<d1b2>
<ehntoo> that would be a pretty good motivator for me to compensate - the ability to decimate the 8GSa/s coming out of this scope a couple times over and still have usable bandwidth was attractive.
<d1b2>
<ehntoo> I should probably run through a self-cal before thinking about anything like that, though. I'm not sure I've actually done one since this unit arrived.
<azonenberg>
We actually have a resolution enhancement filter. You can do the cal yourself by using the average statistic to average a long, few million points, waveform with the input terminated
<azonenberg>
then apply the resolution enhanecment filter and a DC offset filter (order doesn't matter)
<azonenberg>
and manually subtract theoffset
<azonenberg>
Unless we made a fancy automation flow for it, you'd have to manually record the cal constant for each channel, and note that it might change w/ temperature and vertical gain setting on the scope
<d1b2>
<ehntoo> I can definitely live with that.
<d1b2>
<ehntoo> uh, wow. the self-cal made the situation significantly worse. a 50-ohm terminated input is now significantly more than 1LSB out. guess I'm definitely compensating in software then. 😅
<azonenberg>
lol
<azonenberg>
and this is the difference between a $400 Rigol and a $40K LeCroy
<azonenberg>
:p
<d1b2>
<ehntoo> oh yes. it's definitely more than a little rough.
<azonenberg>
Siglent has seemed, to me, to be the "premium entry level" brand these days
<azonenberg>
For when you have somewhat limited budget but want something nicer than the cheapest thing on the market
<d1b2>
<ehntoo> Siglent is what I've recommended to a couple friends that have asked recently, especially after my experiences with the MSO5074. It's got some capabilities that are nice, and for the $800 I spent there's nothing that will quite compete, but it can bite you if you're not careful. Not a desirable attribute for most people, I just happen to be a glutton for punishment.
<azonenberg>
Yes. I no longer recommend rigol to anyone
<azonenberg>
I make Siglent's limitations clear, they are not without fault
<azonenberg>
But they are a good value for the price
<azonenberg>
New mobo for VM server came in, going AFK to install