<_whitenotifier-5>
[scopehal] azonenberg d11ee68 - GetValueAtTime: don't crash if waveform is non-null, but has no samples in it
<_whitenotifier-7>
[scopehal] azonenberg 857e597 - Merge branch 'master' of github.com:ngscopeclient/scopehal
<_whitenotifier-7>
[scopehal] azonenberg ff20047 - Finished initial implementation of PAMEdgeDetector. Still needs work but at least somewhat functional. Fixes #77.
<bvernoux>
It is why I have created mingw-bundledlls
<d1b2>
<david.rysk> again are you downloading the artifact? because it runs on my machine
<d1b2>
<david.rysk> I haven't worked on this yet
<bvernoux>
as there is no any existing things which does that recursive dependency check for Windows DLL/EXEZ
<bvernoux>
ldd does not work for that of course ;)
<bvernoux>
the issue is on a clean PC without all mingw64 bin in path
<d1b2>
<david.rysk> ah yeah
<d1b2>
<david.rysk> I have not worked on that yet
<bvernoux>
good practice is to do not add mingw64 bin path to default
<d1b2>
<david.rysk> at the moment it's assumed you're self compiling
<d1b2>
<david.rysk> is cmake's GET_RUNTIME_DEPENDENCIES incomplete?
<bvernoux>
yes cmake is crap for that
<bvernoux>
it never worked
<bvernoux>
like I said nothing exist on Windows EXE/DLL it is why I have wrote mingw-bundledlls
<d1b2>
<david.rysk> well it wouldn't have worked because we weren't finding dependencies properly
<d1b2>
<david.rysk> that should be fixed now
<d1b2>
<david.rysk> anyway I'll look into it
<bvernoux>
I doubt it will find all dependencies ;)
<bvernoux>
It shoul be too clever for that doing recursive find on all DLL & EXE
<d1b2>
<david.rysk> if it doesn't find all dependencies, then either we aren't finding dependencies properly or there are upstream bugs in .cmake files
<d1b2>
<david.rysk> it shouldn't be doing recursive find — rather, the dependencies we add include lists of required dlls
<bvernoux>
For me knowledge of Cmake it is broken by design ;)
<bvernoux>
me->my
<d1b2>
<david.rysk> cmake has a weird language but is extremely predictable
<bvernoux>
Each time I'm using CMake it is a huge pain, I think you had lot of pains to cleanup all that mess too
<d1b2>
<david.rysk> nah, it was pretty easy, but I've worked with cmake before
<d1b2>
<david.rysk> definitely beats autotools
<d1b2>
<david.rysk> or scons
<bvernoux>
autotools is pure crap
<d1b2>
<david.rysk> autotools is great if you don't care about windows and need to support legacy platforms 😉
<d1b2>
<david.rysk> scons is crap
<d1b2>
<david.rysk> it's not flexible enough gor all too many use cases
<d1b2>
<david.rysk> for*
<d1b2>
<svenpeter42> Sure, I can try on the weekend. Feel free to remind me if I forget 🙂
<bvernoux>
So yes at end only CMake seems a good option for multiplatform, it is just I hate how it work and I'm not using it often
<d1b2>
<david.rysk> a lot of people like meson but it has its own limitations
<bvernoux>
I'm trying to rebuild all from mingw64
<bvernoux>
So far nothing have changed for the build
<d1b2>
<david.rysk> I updated the manual
<d1b2>
<david.rysk> but the pdf might still be old
<d1b2>
<david.rysk> you shouldn't need the Vulkan SDK
<d1b2>
<david.rysk> you should just have to install a set of packages and then cmake / make
<bvernoux>
I really prefer to use the SDK
<d1b2>
<david.rysk> then you'll have to wait for MSVC support which I'm working on
<azonenberg>
Yes. The rendered downloadable PDF/HTML docs on the website are a few weeks out of date
<bvernoux>
else it will mix up different version of mingw64 and can trig very bad issues
<d1b2>
<david.rysk> the SDK is build against MSVC 2015+
<d1b2>
<david.rysk> so you can't link the mingw/ucrt/clang64-built binaries against the SDK's DLLs
<d1b2>
<david.rysk> the only thing that gets used from the
<d1b2>
<david.rysk> the only thing that gets used from the SDK is glslc*
<bvernoux>
the most important for build is to do something predictive it is why I hate using lib depending from pacman
<d1b2>
<david.rysk> (with the old instructions)
<d1b2>
<david.rysk> again, MSVC C++ library linkage is incompatible with MSYS
<bvernoux>
it works in fact with mingw64
<d1b2>
<david.rysk> no it doesn't, the old instructions had certain components installed from the repositories
<bvernoux>
even if it is not optimum
<d1b2>
<david.rysk> or... no, you were building some components from scratch
<d1b2>
<david.rysk> and the Vulkan C++ headers are header-only
<d1b2>
<david.rysk> (and call C API)
<d1b2>
<david.rysk> anyway the proper "fix" for that is to use msvc
<d1b2>
<david.rysk> but that needs more work
<bvernoux>
for some stuff that was using SDK with lib built for VisualStudio
<bvernoux>
and it worked by chance
<bvernoux>
but yes the best it to rebuild all in theory
<d1b2>
<david.rysk> if it's C APIs then it will be fine
<bvernoux>
or to use VisualStudio for the SDK for Windows
<d1b2>
<david.rysk> if it's C++ APIs, not at all
<bvernoux>
Yes I'm aware of crap stuff with C++ ABI
<bvernoux>
which always break between version compiler ...
<d1b2>
<david.rysk> well MS changed that finally, though it's not ideal
<d1b2>
<david.rysk> MSVC 2015 and later use a unified C++ ABI
Stephie has quit [Quit: Fuck this shit, I'm out!]
<d1b2>
<david.rysk> it's not ideal because they're held back from implementing certain newer C++ features
<d1b2>
<david.rysk> I'm not going to tell end-users to build a bunch of unnecessary stuff
<d1b2>
<david.rysk> I'd rather tell them to use the repository packages and we deal with breakage (either by fixing it when CI catches it, or they file a bug)
<d1b2>
<david.rysk> the usage instructions need to be straightforward
<azonenberg>
yeah almost every new windows user has run into problems using the old build flow
<bvernoux>
azonenberg, Except me I have provided the documentation but yes it was a big mess to rebuild lot of things
<bvernoux>
anyway thanks to azonenberg to have switched to Dear ImGui instead of the crappy GTK ;)
<balrog>
Qt would have been better than GTK
<balrog>
ImGui is good but has its own compromises
<bvernoux>
GTK is hell especially for App like ngscopeclient ;)
<bvernoux>
balrog, Qt is crap with their licensing
<balrog>
how so? It's LGPLv3
<balrog>
just like GTK
<bvernoux>
balrog, it is full of slowness crap in the background too for lot of class
<balrog>
Yeah there are certain portions that are GPL but you don't have to use those
<balrog>
that often comes down to how you use it
<balrog>
e.g. if you use QtWebEngine heavily, sure
<bvernoux>
balrog, it is a total mess to use Qt in lot of context even more since their v6
<bvernoux>
balrog, all their new feature related to Web .. are bloated
<bvernoux>
balrog, ImGui is clearly the best
<balrog>
ImGui has a lot of compromises, e.g. it doesn't use system widgets etc
<bvernoux>
balrog, Usually system widget are slow ImGui is to do more advanced/faster stuff
<balrog>
If I were going from scratch I'd be looking at Qt or React Native
<balrog>
GTK+ would be out of the question, as well as wxWidgets
<bvernoux>
balrog, both are bad things for an application like ngscopeclient which shall display tons of data with fast UI
<bvernoux>
balrog, there is no any other GUI available which can handle that, I do not even speak about list ;)
<balrog>
yeah well you don't use those frameworks for displaying the data, you make a window and then render to it directly
<balrog>
heh
<bvernoux>
balrog, the events are awfully slow in Qt too
<bvernoux>
you need to force refresh what a crap
<bvernoux>
it is good for standard UI to do fast crap but not for advanced stuff
<balrog>
that's more an issue of how it's being used than anything
<bvernoux>
at the end if it is to rewrite all like I said ImGui is really better ;)
<bvernoux>
The docking is amazing in ImGui find something like that in Qt ;)
<balrog>
we'll see if the compromises of ImGui cause serious problems, I don't think we're at the point where we can tell yet
<balrog>
I find the docking to be unintuitive and annoying
<balrog>
but that's me
<balrog>
I wish I could turn it off completely
<bvernoux>
For complex UI like ngscopeclient I do not see how we can handle that without docking
<balrog>
I prefer floating windows
<bvernoux>
you can use both anyway with ImGui
<balrog>
yeah except when I drag a window into the wrong place, it docks and I don't want that
<bvernoux>
docking shall be disabled for that
<bvernoux>
balrog, a good example of why Qt is not good for fast UI check PulseView
<bvernoux>
you will understand
<bvernoux>
I was thinking like you at start that Qt is the best for that
<bvernoux>
but when I have discovered ImGui it is night and day even if that requires some hard work for some stuff
<bvernoux>
balrog, no I do not have any Saleae HW, as I use DSLogic U3Pro16 with DSView
<bvernoux>
DSView is based on PulseView (lot of things rewritten) and it reuse all sigrok decoders
<d1b2>
<johnsel> honestly QT is a joke for high performance. Imgui is far more useful and flexible. It is very extendable. Many games use it to render their menus. I'm sure it's possible to add a setting to disable docking completely
<d1b2>
<johnsel> We had the same discussion with GTK, azonenberg said "it doesn't matter because the high performance stuff renders to a window and is independent from the window system", but the reality is it is not. There are always synchronization points for vsync or events or just basic rendering which are not built for 60+hz update rate with a few gigabytes of data. ImGUI is written for game engines, which are exactly that, 60+hz refreshing with lots of
<d1b2>
gigabytes of data being rendered to screen.
<d1b2>
<johnsel> It is written to quickly render an interface and get out of the way of the main process, and it does that pretty well after these many years
<d1b2>
<johnsel> I won't even start about React Native
<d1b2>
<david.rysk> But wasn't gtkscopeclient using gtk APIs to render?
<d1b2>
<david.rysk> iirc due to how it worked it would have been very difficult to make it use Vulkan
<d1b2>
<david.rysk> glscopeclient*
<d1b2>
<johnsel> some parts, but the main question was "is the vulkan shader held back by gtk" and the answer was a resounding yes
<d1b2>
<johnsel> anyway it used gtk for the GUI with a separate texture with the main compute shader written in Vulkan
<d1b2>
<johnsel> looking at the logs for specifics; we had GTKGLArea to host the vulkan shader
<d1b2>
<johnsel> The simplest way to draw using OpenGL commands in a GtkGLArea is to create a widget instance and connect to the GtkGLArea::render signal: The render() function will be called when the GtkGLArea is ready for you to draw its content:
<d1b2>
<johnsel> so GTK tells you when to render, not the other way around
<d1b2>
<johnsel> so you're limited in performance by GTK, which isn't written to deal with our requirements
<d1b2>
<johnsel> furthermore, since we run the filter graph when the shader renders a new frame, we lose even more performance on this side
<d1b2>
<johnsel> you'd want to execute that code as fast as you can ideally at 60hz or above
<d1b2>
<johnsel> it's not the best way of doing things, ideally you separate your data layer from your presentation layer and have the shaders be a window into memory that is produced by another thread that runs seperate from the rendering process imo
<d1b2>
<johnsel> either way you lose performance having a GUI framework that has component model where it decides when to render what and you get callbacks to do your work
<d1b2>
<johnsel> you want direct top level control of the data rendering process
<bvernoux>
Thanks johnsel for your amazing summary ;)
<bvernoux>
ngscopeclient is clearly using the best available technology today to last lot of years ...
<balrog>
well there are good reasons why GUI toolkits throttle refresh
<d1b2>
<johnsel> We don't want the GUI toolkit to throttle anything. It should produce a GUI and handle input events and do this with as little load as possible and while staying out of the way of the main process as much as possible. The trend in applications that deal with lots of data, of which games pretty much lead the charge, is to use lower level APIs and take more control. Andrew picked Vulkan, amongst other reasons, because of that. It's not feasible
<d1b2>
to then introduce another high level GUI framework that has all sorts of legacy code, design principles and performance expectations on the level of your average data entry application with some minor plotting. Many react based plotting libraries (and I'm sure GTK will be at the same level) are happy that they can plot a million points real-time, that's their best. That's 1Msa/s, We deal with gigasamples and Andrew even has a 40 Gigasample scope. That's
<d1b2>
40 000 000 000 as opposed to 1 000 000 points. It's really not even in the same league. To do this properly you should look at HPC and state of the art game rendering, not something people use on a phone or build data entry apps with
<d1b2>
<johnsel> I'm not saying it's not possible by the way, many if not most scopes nowadays do something similar. But then they only have to support 1 specific piece of hardware, and even they have been introducing things like fpga or asic accelerated rendering to offload the problem to something that is capable
<d1b2>
<david.rysk> You wouldn't want to use the GUI toolkit to plot points, you would want to bypass it and render directly into a framebuffer
<d1b2>
<johnsel> We find that a modern GPU is plenty capable, and so you should optimize for that
<d1b2>
<johnsel> No you introduce some kind of widget that then does things for you, but you still need to integrate it with the rest of your platform. Also note cursors, text overlays etc. At some point you're building your own GPU accelerated GUI framework anyway
<d1b2>
<david.rysk> If anything my biggest concerns about imgui aren't due to how it works, but due to it being a one-person project that has very limited financial support
<d1b2>
<johnsel> I've followed and used the project for many years before we started using it for ngscopeclient. It has been consistently used and funded over the years
<d1b2>
<vipqualitypost> is it really just ocornut or are there other "main" devs too? I am under the impression it is just them, and that struck me as surprising for the size of the project.
<d1b2>
<johnsel> primitiveToGeometry
<d1b2>
<johnsel> it's not just him
<d1b2>
<johnsel> it's definitely mainly him though
<d1b2>
<johnsel> 6444 commits compared to 108 the next one down
<d1b2>
<johnsel> with 1000 PRs a decent amount of them might be external contributions after all
<azonenberg>
ok so a few things to note... frist off, one of the problems with glscopeclient and GTK was that GTKGLArea has a separate opengl context for every widget
<azonenberg>
which means, if you want to have gtk chrome around 20 plots
<azonenberg>
you have 20 separate opengl contexts and switching between them is a very heavyweight operation
<d1b2>
<johnsel> ah yes I forgot about that too
<azonenberg>
Which is why imgui was great because we could bypass all of that. As far as rate limiting UI interactions go, we actually support a "power optimized" event loop that runs at a constant low frequency (4 Hz by default) or when a UI event occurs
<azonenberg>
the problem is that right now there's no way to force a cycle of the event loop when a waveform comes in
<azonenberg>
That's something i want to look at improving
<azonenberg>
Waveform rasterizing and the filter graph already run in background threads but there are more sync points between those and the UI thread than I would like
<azonenberg>
Which currently limits the max acquisition rate to the display refresh rate because we cant render >1 waveform per video frame
<d1b2>
<johnsel> Yes decoupling presentation from from data would be a good thing
<azonenberg>
It's most of the way done but the last bits are going to be tricky
<azonenberg>
For example, right now we assume that each stream has a "current" waveform
<azonenberg>
and every step in the filter graph and rendering expects to operate on that
<d1b2>
<david.rysk> it looks like in Qt you can create a QVulkanInstance and then associate it with windows as needed, but I'm not sure how you'd handle rendering different content. (imgui is fine for now; note that you can also give it an existing VulkanInstance)
<azonenberg>
but that makes a lot of parallelism tricky to extract, i really want to be able to have a pipeline of multiple waveforms in flight at once
<azonenberg>
anyway, back to debugging this history data corruption issue
<d1b2>
<johnsel> Yes would need some kind of controller to handle that instead of the graph itself owning the process
bvernoux has quit [Quit: Leaving]
<azonenberg>
Yeah it would also just in general be a pretty sweeping refactoring
<azonenberg>
So its something i'd think about after fixing some nearer term stuff
<azonenberg>
I do see that as being necessary for the long term health of the project as we start moving towards the goal of being able to render and decode tens of Gbps of sample data in real time
<d1b2>
<johnsel> For sure, it's another whole revision of the data model
<d1b2>
<johnsel> You're reinventing MVP 🙂
<azonenberg>
i mean we already sorta have that already, WaveformArea is the view, the Session is the controller, and the filter graph is the model
<azonenberg>
its not exactly 1:1
<azonenberg>
and theres definitely more interdependencies than i'd like in some spots
<d1b2>
<johnsel> Well the point of MVC is to have clear seperation of concern. So you'd really have very clean cuts between presentation and model and controller.
<d1b2>
<johnsel> so the filter graph would exist as a graph, but the logic to traverse that graph and execute whatever is a concern of the controller
<azonenberg>
That's FilterGraphExecutor
<azonenberg>
Which is already a separate object
<azonenberg>
The spaghetti-ish bit happens more around GUI stuff which is always tricky
<azonenberg>
in particular with things like cursors and protocol decode overlays that have to be rendered based on current data
<d1b2>
<johnsel> Yeah parts of it are there, but to really cleanly cut the GUI from the data and from controllers needs some work
<azonenberg>
Yeah. On the longer term todo
<d1b2>
<johnsel> and the controllers might be a little more well thought out
<azonenberg>
The bigger issue architecturally is just the concept of a "current" waveform for a channel probably is not sustainable
<azonenberg>
it will have to be a more explicit dataflow model where the data is in parallel with vs attached to the channel
<azonenberg>
and you can have multiple waveforms that belong to a channel at different points in time
<d1b2>
<johnsel> I agree, I brought that up a long time ago but I never got it explained well enough
<azonenberg>
It's not going to be simple or easy so it keeps getting kicked down the road :p
<azonenberg>
we have bigger and higher priority refactorings because this is not yet the big bottleneck
<d1b2>
<johnsel> Yeah I'm hoping I can start contributing more on the code side
<d1b2>
<johnsel> I'm currently looking at the cairo stuff
<d1b2>
<johnsel> seeing if I can rewrite that to a shader and do depth testing
<d1b2>
<david.rysk> yeah that would be very useful, right now building on msvc (using vcpkg) would do, but cairomm pulls in like an hour of build dependencies
<d1b2>
<johnsel> I think that would be useful if I can deal with some of those annoying bits
<azonenberg>
Great, yeah cairo is a pretty big dependency since it pulls in a lot of libs we dont really need
<d1b2>
<johnsel> Yeah exactly, and in principle it's not that difficult what we need to do
<azonenberg>
Doing it in vulkan is an option but doing it with a simple flood fill algorithm in software would probably be easy enough too
<d1b2>
<johnsel> it's just a matter of understanding the current code since it does a lot of conversions
<azonenberg>
it absolutely doesnt have to be accelerated
<azonenberg>
it's rendered once every time the eye changes size by more than a factor of two
<d1b2>
<johnsel> I'll keep that in mind
<azonenberg>
or when we change the mask we're testing against
<azonenberg>
The *output* of the mask test is checked every filter graph evaluation
<azonenberg>
the rasterized mask i mean
<d1b2>
<johnsel> It'd be nice to contribute a shader though since then I touch most big parts of the application
<azonenberg>
but the generation of the raster mask is extremely non critical
<azonenberg>
Yeah it might bring in more baggage than you think though
<d1b2>
<johnsel> Yeah.... haha
<azonenberg>
because right now we do not really do any vulkan geometry ourselves
<azonenberg>
its all through imgui
<azonenberg>
the only shaders we explicitly invoke are compute
<azonenberg>
then all of the custom rendering is either lines and text via ImDrawList
<d1b2>
<johnsel> Yeah I have to see if it's viable to add it in the current model
<azonenberg>
or textured triangles via ImDrawList
<azonenberg>
so adding all the infrastructure to make vulkan rendering pipelines for this one thing seems like a horrible idea
<d1b2>
<johnsel> hmm that could be an option potentially
<azonenberg>
imdrawlist is meant to draw to on screen
<d1b2>
<johnsel> assuming imgui gives me what I need to depth test
<azonenberg>
i dont think it'd be the right tool for this job
<d1b2>
<johnsel> I'll have to check
<d1b2>
<johnsel> and yes if I need a whole rendering pipeline I'm not going to do it
<d1b2>
<johnsel> I wouldn't be able to do it properly anyways
<d1b2>
<johnsel> I'm not sure how much effort it is though so I'll just see how far I get
<d1b2>
<johnsel> and probably end up with a sw flood fill anyway
<azonenberg>
yeah this really seems like it needs a simple flood fill algorithm and nothing more
<_whitenotifier-5>
[scopehal-apps] azonenberg cdeb7fc - Merge pull request #691 from d235j/fix-font-scale Fix default font not changing when font is already loaded in atlas
<_whitenotifier-5>
[scopehal-apps] azonenberg cac1b21 - Session: load markers after waveform data to avoid stepping on things and leading to out-of-order history entries and corrupted waveform content
<d1b2>
<johnsel> @david.rysk I'm currently on an older commit still with the old build process
<d1b2>
<johnsel> Assuming I would pull latest, what would you recommend for me to get a debug build with a debugger attached? Can I simply tell vs-code to use the CMakeLists and mingw64?
<d1b2>
<david.rysk> what debugger are you using? So far I've been using gdb in the mingw/ucrt/clang shell
<d1b2>
<david.rysk> vs-code doesn't have a debugger of its own, it's probably using something else under the hood
<d1b2>
<johnsel> none actually, haven't had the need for it since I haven't really touched much code
<d1b2>
<johnsel> but gdb is fine
<d1b2>
<johnsel> anyway reading between the lines the expectation is that it should work fine and I can point vs code to attach gdb
<d1b2>
<johnsel> here I go then
<d1b2>
<david.rysk> if vs code uses gdb then it should work fine
<d1b2>
<johnsel> is main master fine?
<d1b2>
<david.rysk> yes
<d1b2>
<johnsel> or are we still on one of y, ahh
<d1b2>
<johnsel> oki let's see
<d1b2>
<david.rysk> oh do you have a HiDPI monitor?
<d1b2>
<johnsel> nein
<d1b2>
<david.rysk> there are still bugs around HiDPI on windows that I have to track down
<d1b2>
<johnsel> well I do have one but not using it
<d1b2>
<johnsel> I'm using my trusty old 2560x1080 widescreen monitor
<d1b2>
<johnsel> which is probably one of the first things I will upgrade next
<d1b2>
<david.rysk> 2560x1080 isn't bad if it's small enough
<d1b2>
<david.rysk> I have a 2560x1440 27" that I'm very happy with
<d1b2>
<johnsel> This is 34" or so, so pretty much 27" but wider
<d1b2>
<johnsel> @david.rysk jamms@JOHN-PC MINGW64 ~/scopehal-apps/build $ cmake .. -- Building for: Ninja --snip-- -- Build files have been written to: C:/msys64/home/jamms/scopehal-apps/build jamms@JOHN-PC MINGW64 ~/scopehal-apps/build $ make make: *** No targets specified and no makefile found. Stop.
<d1b2>
<johnsel> alright cmake --build . does work though
<d1b2>
<johnsel> hey can we do a #define message(ignore) or something to suppress those pragma messages from yaml-cpp