<_whitenotifier-7>
[scopehal-apps] ehntoo da5a908 - Add feature detection to power supply control
<_whitenotifier-7>
[scopehal-apps] ehntoo 80d42d4 - Correct ngscopeclient current apply button control
<_whitenotifier-7>
[scopehal-apps] ehntoo 0144df4 - Modify libpng CMakeLists includes This improves compatibility with macOS and should still work fine on Windows/Linux.
<_whitenotifier-7>
[scopehal-apps] ... and 2 more commits.
<azonenberg>
ehntoo: ok so are you actively working on anything at the moment now that you got the power supply stuff merged?
<d1b2>
<ehntoo> I was going to give glscopeclient a go on macOS and see if it's working now that the vulkan renderer has landed. Apart from that, I'll be seeing about improving stability with my rigol mso5000 - I hit quite the variety of crashes in driver code the other day.
<azonenberg>
Lain is actively working on getting glscopeclient to work on macos right now
<azonenberg>
probably best to give it a day or two so you dont waste time both fixing the same bugs
<d1b2>
<ehntoo> sounds good.
<lain>
ok at this point my macos dev branch builds both ngscopeclient and glscopeclient
<lain>
ngscopeclient fails immediately with "ERROR: glfwGetRequiredInstanceExtensions failed", which iirc is likely because I need to reduce the required gl version?
<lain>
glscopeclient fails with "ERROR: vk::SystemError: vkCreateInstance: ErrorIncompatibleDriver"
<azonenberg>
lain: hmmmmm
<azonenberg>
glfw is supposed to be operating in vulkan only mode in ngscopeclient
<lain>
ah, hm
<azonenberg>
it should not be initializing opengl at all, ngscopeclient is pure vulkan
<azonenberg>
glfwGetRequiredInstanceExtensions() should be returning the list of *vulkan* extensiosn that we need to create a moltenvk context
<azonenberg>
extensions*
<azonenberg>
ehntoo: did you have to do anything special to get moltenvk to work for you?
<azonenberg>
what platform were you testing on?
<lain>
oh, this might be my mistake
<d1b2>
<ehntoo> I installed the vulkan SDK DMG from https://vulkan.lunarg.com/sdk/home. I don't immediately recall having to install anything else for vulkan support
<d1b2>
<ehntoo> my testing has been done on a 14" m1 max macbook pro
<lain>
ehntoo: what did you set the VULKAN_SDK environment variable to?
<lain>
or did you choose a whole-system install during the installation
<d1b2>
<ehntoo> I did a whole-system install
<lain>
hm ok
<lain>
azonenberg: are there any gcc-isms I need to be aware of in the codebase? cmake defaulted to building with clang, but I can force it to gcc if you think that's necessary
<lain>
though I'd expect it to fail compilation if that were the case
<d1b2>
<ehntoo> I've been using clang 14 from homebrew so far.
<d1b2>
<Darius> clang knows most gccism anyway
<d1b2>
<david.rysk> gcc is not gonna cut it on macOS
<lain>
that's what I figured
<d1b2>
<Darius> uh why?
<d1b2>
<david.rysk> You run into runtime incompatibility issues
<d1b2>
<Darius> you do?
<d1b2>
<david.rysk> At least whenever I’ve tried
<d1b2>
<Darius> absolutely not my experience
<d1b2>
<Darius> macports builds a number of things with GCC that seem to work fine
<d1b2>
<ehntoo> I had issues with gtk and c++ ABIs when trying to use GCC on macOS.
<d1b2>
<david.rysk> Maybe macports fixed its GCC to use libcxx — I dunno
<d1b2>
<david.rysk> There’s also the total lack of support of ObjC 2.0
<d1b2>
<david.rysk> Yeah, cxx ABI incompatibility is mainly what I’ve run into
<d1b2>
<david.rysk> If all you’re doing is C ABI you’ll be fine
<d1b2>
<Darius> ahh
<d1b2>
<Darius> yeah only C fo rme
<lain>
hmm it looks like my vulkan installation has some issues
<lain>
let me try reinstalling it, not sure what could've gone wrong o.o
* monochroma
has flashbacks to GCC and MIPSpro ABI weirdness on IRIX
<lain>
lol
<lain>
HOKAY
<lain>
now ngscopeclient works
<lain>
apparently not doing the global installation for Vulkan SDK was a bad idea, I'll have to chase down why some other day, but that's low priority
<lain>
and I have ngscopeclient rendering waveforms from the demo scope on macos
<lain>
huzzah
<lain>
now let's see about glscopeclient...
<lain>
ok glscopeclient comes up but when I add the demo scope it errors out, looks like an easy fix though
<lain>
ehh ok, that's gonna be more work than I have time for tonight, but yay everything building on macos!
<d1b2>
<ehntoo> progress!
<lain>
indeeeed! :3
<d1b2>
<david.rysk> Is there any point in testing on a mac yet?
<lain>
I'll confirm in the coming days, but this should also mean we have full arm64 linux support
<d1b2>
<david.rysk> (Meaning — is there a lot of MoltenVK integration that still needs to happen?)
<lain>
david.rysk: there's a fair bit of work to be done still yet, I've been tasked with the macos port. specifically for Apple Silicon, but I'll make sure it works on x86_64 while I'm at it
<lain>
now that the renderer is completely vulkan, though, it should be straightforward
<d1b2>
<ehntoo> ngscopeclient already works pretty well under macOS on my m1 machine with the immediate-mode render that was merged recently. It's not fast yet, but it seems to work.
<azonenberg>
yeah. Tonight's task after fork(2) goes to sleep is to begin gluing the compute shader vulkan renderer into ngscopeclient
<azonenberg>
hopefully i'll have that working later tonight or tomorrow
<azonenberg>
And then after that i can work on making ngscopeclient more feature complete
<azonenberg>
meaning cursors/markers, history, file load/save, and filter graph support
<azonenberg>
and file export and (gpu accelerated) multiscope sync
<azonenberg>
those are afaik the main todo items holding us back from feature parity w/ glscopeclient now
<azonenberg>
and we already have a few things (power supply/RF signal generator support and much better window management) that it doesn't have
<lain>
woo
<d1b2>
<ehntoo> to quantify "pretty well" for performance with what's already together, a RelWithDebInfo build of lain's macos branch, I get 80Hz rendering on my m1 max machine according to the Performance Metrics
<d1b2>
<ehntoo> lost a bit of that sentence during an edit - that's 80hz of the demo scope view
<azonenberg>
and whats the waveform update rate?
<azonenberg>
is that 80 WFM/s or just 80 FPS with waveforms updating slower?
<azonenberg>
Anyway, awesome to see things coming together :)
<d1b2>
<ehntoo> ~37.5 Hz waveform update rate, but I'm not sure what the bottleneck is there. the "Pending waveforms" metric seems to alternate between 0 and 1 on alternating frames
<azonenberg>
Yes. That means you're rendering faster than the scope can supply waveforms, which is to be expected
<azonenberg>
The demo scope is not speed optimized
<azonenberg>
it does a lot of mersenne twister math and such to generate nice smooth gaussian noise on the waveforms
<azonenberg>
and on the x86 version also even does a fft convolution to apply a dsp lowpass filter
<azonenberg>
but the arm64 version has that disabled because FFTS doesnt seem to work on mac
<azonenberg>
and so far nobody has bothered to convert that to use vkfft
<azonenberg>
An actual scope driver built for speed, like a picoscope, should easily be able to get way faster than that
<azonenberg>
i dont think pico has native mac drivers? but if you run ps6000d on a linux (or windows? idk if anyone has tried that) box you can connect over lan
<azonenberg>
at which point your bottleneck is likely to be the network pipe, unless you have a thunderbolt 10GbE dongle on the laptop
<azonenberg>
lain: ok so making some progress. I have your shader copied to ngscopeclient and building under cmake there (trivial copy paste from the glscopeclient cmakelists)
<azonenberg>
and i'm starting to work backwards from final rendering, making a vulkan texture that i can display the waveform in
<azonenberg>
and beginning to lay the groundwork for the tone mapping shader
<azonenberg>
which means adding support for samplers to the ComputePipeline class
<azonenberg>
or images, sorry
<azonenberg>
it has to be able to output to a texture
<azonenberg>
so i have to figure out the proper vulkan way to bind a writable image as an output resource
<azonenberg>
Not going to finish tonight but it's coming along
<lain>
:D
bvernoux has joined #scopehal
<bvernoux>
I was just trying latest glscopeclient with Vulkan Render but on my computer it is clearly slower now
<bvernoux>
lost about 4fps
<bvernoux>
also the FFT have a bug now
<bvernoux>
and the waveform rendering is strange too like pixelated
<bvernoux>
it is like there is not enough segment/point on the demo vs previous version
<bvernoux>
the FFT bug is towards the peaks which are not centered on the waveforms
<bvernoux>
it is fixed if I change horizontal scale on waveform
CobaltBlue has quit [Ping timeout: 244 seconds]
<azonenberg>
Resize broken i think is a known bug lain is working on
<azonenberg>
There is a redundant GPU-CPU-GPU copy right now that slows things down. with ngscopeclient that won't be needed
<azonenberg>
There are some potential alpha blending issues we're looking into
<azonenberg>
as far as the offset issue, interesting. that is not a known problem
<bvernoux>
could you reproduce the offset issue ?
<bvernoux>
with FFT
<bvernoux>
it was not present before of course
<azonenberg>
Looking...
<azonenberg>
yes, i confirm what appears to be a horizontal shift of the image
<azonenberg>
lain: poke re above ^
<azonenberg>
it's not just FFTs, if you do a cursor or something you can see that the text is displaying the edge slightly off where it's rendered
<azonenberg>
conjecture: it was related to your width vs plotwidth thing
<azonenberg>
maybe size per pixel is slightly incorrect
<azonenberg>
confirmed, it appears to be a scaling problem
<azonenberg>
the shift between rendered and actual position of an edge gets greater as you move towards the right of the image
<azonenberg>
and it's related to pixel position within the plot, not sample timestamp within the waveform
<azonenberg>
when i use the demo scope, if i put the start of the waveform at the very left of the image, the waveform starts right about time zero
<azonenberg>
as i drag the timebase to move the 0fs point on the timeline further and further right in the image
<azonenberg>
the rendered waveform moves right faster such that it begins at a positive time offset
<azonenberg>
at a quick glance, when the waveform is at the very right edge of the plot, and thus the error is maximal
<azonenberg>
the shift appears to be about as wide as the Y axis area
<azonenberg>
So it looks like you're rendering the region of time corresponding to the entire plot including Y axis, and then squishing that texture into the space left of the Y axis
<bvernoux>
On my side I'm trying to design/find good fixture for Dual 4mm Banana plug to SMA ;)
<bvernoux>
So far the max frequency reached with -3dB is about 400 to 500MHz
<bvernoux>
on one side there is a SMA and on other side Dual Banana Plug with Adapter to SMA
<bvernoux>
and we can say between the SMA and the Dual Banana Plug there is 1cm wire
<lain>
aha
<lain>
I'll get that fixed either later today or tomorrow
<azonenberg>
Great. I'm back to working on the shader port in ngscopeclient
<bvernoux>
yes anyway good job as it works even on Windows ;)
<bvernoux>
and the bugs found are not specific to windows too
<bvernoux>
Also maybe it will be great to load the new waveform-compute.* from a directory
<bvernoux>
like shaders
<bvernoux>
as so far they are expected to by in same directory as glscopeclient
<bvernoux>
I have not really understood why the CI Build fail with a strange file name ...
<bvernoux>
which does not match with what is generated
<bvernoux>
This file name is wrong waveform-compute.analog.dense.int64.spv
<bvernoux>
What is generated is waveform-compute.digital.int64.dense.spv
<bvernoux>
But I have not found where this wrong name was "waveform-compute.analog.dense.int64.spv" built as in CMakefile all seems ok
<azonenberg>
there should be analog and digital shaders generated
<azonenberg>
the dense.int64 vs int64.dense mixup is interesting
<azonenberg>
or is that just a typo on your part?
<azonenberg>
there are dense, analog, and histogram rendering shaders
<azonenberg>
with and without int64 support
<bvernoux>
it a a typo somewhere when build install on CI Build
<bvernoux>
I confirm the scopehal-apps\build\src\glscopeclient\cmake_install.cmake line 57 have that
<bvernoux>
It shall be somewhere in Cmake
<azonenberg>
everything should end up in the shaders dir. but this is brand new code from yesterday so some ruogh edges are expected
<bvernoux>
yes
<bvernoux>
path is wrong too ;)
<bvernoux>
it is missing shaders/
<azonenberg>
i'm also in the middle of trying to get the same code to build in ngscopeclient
<bvernoux>
and the fun is if it is not found we have an ugly exception/crash after the trace which output file not found
<bvernoux>
ha yes the bug seems to be in scopehal-apps/src/glscopeclient/CMakeLists.txt => render_shader_variant
<bvernoux>
the order is not the same as add_custom_target(
<bvernoux>
rendershaders
<bvernoux>
which expect waveform-compute.analog.int64.dense.spv instead of waveform-compute.analog.int64.dense.spv
<bvernoux>
but it is not really clear how to understand that function(add_render_shader_variant outname)
<lain>
ah
<lain>
yeah if someone more familiar with cmake could fix that function, it would be appreciated
<bvernoux>
it is clearly over complex ;)
<bvernoux>
But it is clearly an issue in function(add_render_shader_variant outname)
<bvernoux>
which does the install which fail with CI Build
<bvernoux>
that does not affect the build just the install
<bvernoux>
woo it is a recursive function ;)
<bvernoux>
we shall just invert the sequence
<bvernoux>
hmm no in fact
<azonenberg>
it's one level of recursion for adding int64 / dense or not, it seems
<azonenberg>
anyway, i'll look at cleaning some of that code up in a bit. have higher priority stuff to work on for the next couple hours
<bvernoux>
ha yes it add all different possible types ;)
<bvernoux>
we see the results in scopehal-apps/build/src/glscopeclient/cmake_install.cmake
<lain>
ohh, oops :3
<azonenberg>
lain: if you can fix that bug and the x axis scale problem i'll take care of moving stuff into shaders/ and work on polishing up the code later tonight
<azonenberg>
the cmake code*
<bvernoux>
Change => std::string shaderfn = "shaders/waveform-compute."; in file scopehal-apps\src\glscopeclient\WaveformArea.h too
<bvernoux>
to add shaders/
<bvernoux>
waveform-compute will be searched in shaders (like other shaders)
<lain>
ah, yes, that is a silly mistake
<bvernoux>
This function(add_render_shader_variant outname) is really not clear
<bvernoux>
why not something simpler which just take full name as input ?
<bvernoux>
like in add_custom_target(
<bvernoux>
rendershaders
<azonenberg>
bvernoux: the idea is that we're compiling the same source file with a lot of different #defines
<lain>
cmake is vexing
<bvernoux>
@azonenberg, yes I understand
<bvernoux>
it is just over complex to do that recursive stuff
<bvernoux>
maybe just keep the function not recursive with fill filename at input
<lain>
yeah, I'm simplifying it now
<bvernoux>
it will be explicit like that
<bvernoux>
Especially as there is not so many variant
<bvernoux>
To have something maintenable in an easy way as Cmake is already a headhach for nothing ;)
<_whitenotifier-7>
[scopehal-apps] lain f9bea30 - Actually complete the merge. Whoops.
<bvernoux>
ha ok perfect
<bvernoux>
so all OpenCL or other OpenGL are fully removed
<lain>
yeah
<bvernoux>
only Vulkan stuff and potentially some CPU rendering for specific filters
<azonenberg>
well not quite
<azonenberg>
there is still GL in glscopeclient for the final compositor
<azonenberg>
but it's like GL 2.x or 3.x
<lain>
ngscopeclient is fully vulkan, glscopeclient still uses OpenGL for ... yeah that
<azonenberg>
no compute shaders
<bvernoux>
ha ok
<azonenberg>
I'm still working on getting the shader based renderer working in ngscopeclient. hoping to get that done today or tomorrow
<bvernoux>
ha yes great
<lain>
other than fixing bugs in vulkanized stuff, my main task now is macOS compatibility, specifically on Apple Silicon
<azonenberg>
Yeah. The vulkan conversion was just the big dependency of that
<bvernoux>
so a great step is reached even if there is some "bugs" to be fixed
<bvernoux>
macOS compatibility is towards Vulkan stuff or mainly GTK ?
<azonenberg>
GTK is mostly a problem for windows but GTK + GL is problematic on macOS and GTK+Vulkan integration is basically nonexistent
<lain>
azonenberg: so it looks like in PrepareGeometry we have m_width in a bunch of places, I guess previously the render shader was doing the full m_width and we were discarding the excess (wasted effort), so now those m_width's need changed to m_plotRight, such as on lines 182 and 183, if I'm understanding correctly?
<bvernoux>
or maybe the remaining OpenGL but IIRC it is not possible to use OpenGL with MacOS...
<azonenberg>
lain: yes, i think so
<azonenberg>
test it :)
<lain>
azonenberg: kk
<azonenberg>
bvernoux: GL 2.x / 3.x and up to i think 4.1 works on macos
<azonenberg>
at least with M1
<azonenberg>
removing use of GL 4.3, i.e. compute shaders, was necessary to get it working on macos
<bvernoux>
azonenberg, as so the plan is to keep that small GL2.x stuff in glscopeclient ?
<azonenberg>
Yes. There is no good path to get full vulkan rendering because GTK doesnt really handle that well
<bvernoux>
ha ok
<azonenberg>
that, and general performance, were the drivers behind the shift to imgui
<azonenberg>
So the intent is to make a clean break with GL and GTK and go all vulkan in ngscopeclient
<azonenberg>
and leave all of the legacy stuff behind
<bvernoux>
ok perfect
<azonenberg>
As well as FFTS and OpenCL
<azonenberg>
there is still a bit of FFTS left, but it's on the list of things to completely get rid of
<bvernoux>
OpenCL is totally removed fro glscopeclient no ?
<bvernoux>
from
<bvernoux>
VkFFT have some opencl include
<bvernoux>
but I suspect it is not used ?
<azonenberg>
opencl is completely removed
<azonenberg>
vkfft supports opencl as an alternate backend
<azonenberg>
we are only using the vulkan backend
<azonenberg>
if it includes opencl headers when not using the opencl backend that's a (minor) upstream bug
<lain>
azonenberg: I'm going to revert to m_width instead of m_plotRight for now and make a note that it would be more efficient to use m_plotRight... quickly changing all the relevant widths to m_plotRight did not fix the problem, and I don't have time to root-cause it right now, but going back to m_width seems to fix it
<azonenberg>
yeah i havent copied the fixes over yet
<azonenberg>
that code is from before the fix existed and was only pushed recently
<azonenberg>
wait what
<azonenberg>
i just looked at your link
<azonenberg>
oh isee
<azonenberg>
you're talking about the other cmake script
<bvernoux1>
yes
<bvernoux1>
the fix of the cmake
<bvernoux1>
53% build...
<bvernoux1>
where is my ryzen9 9750x ;)
<bvernoux1>
63% ...
<bvernoux1>
my PC is always faster the GitHub CI Build ;)
<azonenberg>
lol
<azonenberg>
We're working on fixing that very soon
<azonenberg>
I have the GPU and CPU
<azonenberg>
had problems with the motherboard, waiting on a replacement
<azonenberg>
and working with johnsel to figure out the software/provisioning stack to actually run the builders here
<d1b2>
<ehntoo> azonenberg: I've got a number of Rigol MSO5000 changes I'm working on in https://github.com/ehntoo/scopehal/tree/rigol-mso5000-fixes. What I've got now already fixes the crashes I've seen and takes care of some quality-of-life issues, but I'll have some more coming in the near future. Would you prefer I wait until I'm done making improvements to submit a PR, or should I open one now?
<bvernoux1>
ehntoo ha very nice
<bvernoux1>
ehntoo I was planning to push very big fix to have raw socket for that purpose month ago but you have found an other way to do it
<bvernoux1>
the tricky part is to detect when rigol FW fail to send data depending on what is displayed as it is clearly buggy on Rigol FW side
bvernoux1 is now known as bvernoux
<d1b2>
<ehntoo> I need to disassemble some more of the scope firmware to figure out why the *WAI fixes things, but I have seen none of the waveform truncation or corruption that I was seeing before after tossing it in there.
<bvernoux>
for me in some case *WAI was not enough but it shall improve a lot
<bvernoux>
a fun stuff to avoid crashing Rigol FW is to enable FFT ;)
<bvernoux>
that slow down the UI and SCPI was more stable ...
<bvernoux>
yes Rigol synchro between SCPI & UI / Acquisition is a pure joke full of synchro / bugs
<bvernoux>
At least on Rigol MSO5K ...
<bvernoux>
ehnto ha on my case I was doing the *WAI after WAV:DATA? so maybe you have found a real fix ;)
<bvernoux>
as you are doing it before
<d1b2>
<ehntoo> hopefully. 🙂 I'll still be poking around in the firmware to see if I can understand why it's changing behavior, though.
<bvernoux>
the question is do you have always the erro "Error RigolMSO5000 READ 0x%02X "
<bvernoux>
it is when we hit the bug
<bvernoux>
after WAV:DATA?
<bvernoux>
we are expecting N data and sometimes we receive just 1 data which contains 0x0A
<bvernoux>
like it is the end of data without any data
<d1b2>
<ehntoo> I haven't seen it yet when I have a *WAI prior to the WAV:DATA?.
<bvernoux>
ha interesting
<bvernoux>
do you have tested with 10Mpts ?
<bvernoux>
as more you have pts more easily you reproduce that issue
<d1b2>
<ehntoo> hm, I'll give it a try.
<bvernoux>
the only way to catch that is to do not loop on n data
<bvernoux>
like glscopeclient hal do
<bvernoux>
it is why I had in my local repo a hacked socket to use raw socket
<bvernoux>
but as it was not very clean I have never pushed it
<bvernoux>
I confirm latest glscopeclient work fine
<bvernoux>
also ngscopeclient is clearly smoother vs latest glscopeclient
<azonenberg>
well latest ngscopeclient still is using the immediate mode renderer
<bvernoux>
ha ok ;)
<azonenberg>
if the shader based renderer is slow on your ancient machine, i could plausibly see the immediate being faster
<azonenberg>
but it will deliver much worse graphical quality
<bvernoux>
the quality is very nice so far
<azonenberg>
the longer term plan is to have a range of renderers with various speed/quality tradeoffs
<bvernoux>
I will say ngscopeclient is even better than glscopeclient on the demo
<azonenberg>
the immediate mode renderer has weird visual glitches, displays higher noise than is actually present, scales horribly past few kpoints
<azonenberg>
has no intensity grading
<bvernoux>
ha yes ;)
<bvernoux>
we will see with Vulkan mode ;)
<bvernoux>
I suspect gtk do lot of harm in glscopeclient
<d1b2>
<ehntoo> bvernoux: a 10Mpoint depth at 2Gsps seems stable so far under scopehal+ngscopeclient on my mac with the fixes in that branch
<d1b2>
<ehntoo> I'll leave it running a while and see if it crashes eventually.
<bvernoux>
ha great
<bvernoux>
you can push your fixes I will test tomorrow
<bvernoux>
10Mpt is not realistic anyway but if it work fine at 100k & 1M with glscopeclient it is very interesting
<azonenberg>
i mean 10M is a scale i work at routinely with faster scopes
<bvernoux>
yes the issue with Rigol MSO5K is it became dead slow at 10Mpts