azonenberg changed the topic of #scopehal to: libscopehal, libscopeprotocols, and glscopeclient development and testing | https://github.com/glscopeclient/scopehal-apps | Logs: https://libera.irclog.whitequark.org/scopehal
Degi_ has joined #scopehal
Degi has quit [Ping timeout: 268 seconds]
Degi_ is now known as Degi
jevinskie[m] has quit [Ping timeout: 268 seconds]
whitequark has quit [Ping timeout: 268 seconds]
mikolajw has quit [Ping timeout: 264 seconds]
bgamari has quit [Ping timeout: 268 seconds]
bgamari has joined #scopehal
lain has quit [Ping timeout: 252 seconds]
lain has joined #scopehal
<Bird|otherbox> w32api macros strike again...
<azonenberg> miek, Johnsel: i'm pretty sure i remember we had a special #define or something to enable #zu under mingw
<azonenberg> for %zu*
<azonenberg> grep the source code and do some googling
<azonenberg> it was annoyingly non default but i can't remember specifics
<d1b2> <Darius> __USE_MINGW_ANSI_STDIO?
<azonenberg> that sounds right
<azonenberg> i thought we had it defined globally but we might have missed it somewhere
<azonenberg> Johnsel: ^
<d1b2> <Darius> looks like a mess though, since mingw has to guess what the MS runtime supports
<_whitenotifier-7> [scopehal-apps] azonenberg pushed 1 commit to master [+4/-0/±5] https://github.com/glscopeclient/scopehal-apps/compare/bee4c641efd6...e183b897e8ed
<_whitenotifier-7> [scopehal-apps] azonenberg e183b89 - Initial work on ngscopeclient connection dialog
<d1b2> <azonenberg> I'm loving how easy it is to create nice looking UIs with help tooltips etc in imgui
<azonenberg> doing this in GTK is so much more work
massi has joined #scopehal
<azonenberg> On that note, one thing on my todo list is adding support for GUI metadata in filters
<azonenberg> making it so they can declare help text for inputs and such
<azonenberg> and also things like groups of settings
<azonenberg> vs only being alphabetical like they are now
<d1b2> <dannas> Oh my. Experimental imgui support. It will be very interresting to see where this leads.
<azonenberg> dannas: the intent is for this to be a full rewrite of the frontend
<azonenberg> fixing a lot of things that would be painful to do in the existing GTK frontend
<azonenberg> fairly soon, however, i expect to get gated on lain's shader porting work
<d1b2> <dannas> I haven't used any immediate mode GUI frameworks. I wonder how much you'll have to work around not being able to retain states in some gui components
<d1b2> <dannas> (Atleast that's how I've understood imgui frameworks: That you have to redraw everything on each frame and that your framework won't keep things cached inbetween redraws)
<azonenberg> So i actually find it works better
<azonenberg> so far at least
<azonenberg> i can have the gui element operate directly on a state variable
<azonenberg> and not need to call get/set methods to access value held within it
<azonenberg> There will still be higher level state, for things like what dialogs are currently open
<_whitenotifier-7> [scopehal-apps] azonenberg pushed 1 commit to master [+5/-0/±6] https://github.com/glscopeclient/scopehal-apps/compare/e183b897e8ed...2aa3362f1ad7
<_whitenotifier-7> [scopehal-apps] azonenberg 2aa3362 - ngscopeclient can now connect to a scope. No support for actually doing anything useful with it yet.
<azonenberg> ok so i think the next step will be implementing waveform groups/areas and figuring out how to make stuff like that work
<azonenberg> including splitting, window management, drag and drop, etc
<azonenberg> again, still foucsing on the gui side as we cant draw waveform data in vulkan yet
<azonenberg> They do have a nice histogram widget, however
<azonenberg> which i am thinking of using for histogram type waveforms instead of the shader we currently use
<azonenberg> but i'm not dead set either way
<azonenberg> (they also have a waveform drawing widget but it doesn't scale to the megapoint level we need)
<azonenberg> but histograms, even if they have a lot of samples, are still relatively small
jevinskie[m] has joined #scopehal
whitequark has joined #scopehal
<_whitenotifier-7> [scopehal-apps] azonenberg pushed 1 commit to master [+0/-0/±4] https://github.com/glscopeclient/scopehal-apps/compare/2aa3362f1ad7...e1b69f959ad9
<_whitenotifier-7> [scopehal-apps] azonenberg e1b69f9 - ngscopeclient: Initial fullscreen support
<azonenberg> Starting to fool w/ window management. We're using some undocumented/internal imgui APIs so there might be code churn at some point if those change
<azonenberg> but it shouldn't be too bad
bvernoux has joined #scopehal
massi has quit [Remote host closed the connection]
<_whitenotifier-7> [scopehal-apps] Biswa96 commented on issue #470: Windows CI build is broken despite finding Vulkan SDK - https://github.com/glscopeclient/scopehal-apps/issues/470#issuecomment-1242171275
<_whitenotifier-7> [scopehal-apps] Biswa96 commented on issue #470: Windows CI build is broken despite finding Vulkan SDK - https://github.com/glscopeclient/scopehal-apps/issues/470#issuecomment-1242171786
<_whitenotifier-7> [scopehal-apps] Biswa96 deleted a comment on issue #470: Windows CI build is broken despite finding Vulkan SDK - https://github.com/glscopeclient/scopehal-apps/issues/470#issuecomment-1242171275
Johnsel has quit [Ping timeout: 244 seconds]
Johnsel has joined #scopehal
<azonenberg> So one thing i need to figure out is how to integrate imgui with full custom rendering
<azonenberg> i.e. WaveformArea's
<azonenberg> in particular, this isn't like a game with debug windows
<azonenberg> where i have one background scene then a UI on top of it
<azonenberg> i'll actually have UI features in the imgui windows themselves
<azonenberg> have custom rendered features*
<azonenberg> so i need to be aware of the extents of those windows to draw in them
Johnsel has quit [Remote host closed the connection]
<azonenberg> One possible approach is to have imgui manage all of the draw-to-screen stuff
<azonenberg> and have our rendering shaders output a texture
<azonenberg> and just make the texture be a widget we can place in the frame
<d1b2> <johnsel> That seems the most reasonble approach. Treat it as 'custom imgui widgets', which of themselves have little functionality other than to present a texture created elsewhere
<azonenberg> Yeah. The tricky bit happens if i want to mix the two
<d1b2> <johnsel> and maybe some UX to change whatever it does
<d1b2> <johnsel> mix which two?
<d1b2> <johnsel> An existing widget?
<azonenberg> for example, i'd like to be able to use imgui drag-and-drop to manage moving waveforms between viewports
<azonenberg> which means, ideally, that the viewport window contains both the background image texture as well as the button
<azonenberg> etc
<d1b2> <johnsel> if the widget moves and the widget presents a texture with some coordinates then that will happen by itself, no?
<azonenberg> no the point is there would be widgets on/in the other widget
<d1b2> <johnsel> you just tell 'your code' to generate a texture of size X,Y
<azonenberg> look at glscopeclient irght now
<d1b2> <johnsel> make the widget draw that somewhere
<azonenberg> where a waveformarea contains things like the y axis scalebar and the channel proeprties button
<azonenberg> within itself
<azonenberg> and ideally i would like to make the scalebar and the button be imgui widgets
<azonenberg> but they're drawn on/in the texture area
<d1b2> <johnsel> you can just layer them
<azonenberg> you can stack widgets in a single area?
<azonenberg> this is something i'm still looking into
<azonenberg> i'm sure it can be done
<d1b2> <johnsel> you can layer draw calls on top of each other
<azonenberg> i just dont have the familiarity with the platform to do it yet
<d1b2> <johnsel> the functionality doesn't have to be generic enough that you can put every widget in every widget
<azonenberg> Yeah
<d1b2> <johnsel> you can just have a widget with all functionality in one, that layers some stuff on top of each to provide scope output + UX components
<d1b2> <johnsel> I'd just call that the widget functionality
<azonenberg> yeah. the main thing is i just wanted to use imgui drag and drop
<azonenberg> for things like dragging a waveform into another plot
<azonenberg> and things like that
<azonenberg> so basically i need it to be aware of all of the sub-regions within a waveform view
<azonenberg> the other thing i'm still working on is the size-to-fill bit
<d1b2> <johnsel> right, you'd have to see if that is easily done, but I'm sure some other drag and drop code will exist
<azonenberg> imgui normally has each widget have a defined size, and the container optionally autosizes to fit
<azonenberg> in my case i'd be working backwards
<azonenberg> where the waveform group has a fixed (adjustable) size
<azonenberg> and the plots resize themselves to occupy appropriate fractions of its space
<d1b2> <johnsel> I also saw that there is some platform generic file selection functionality code in one of the linked projects for imgui that has all the debug functionality and which I said we could rip a lot of UI out of
<azonenberg> So there will be some custom window layout code for sure
<azonenberg> We'll worry about that way down the pipe
<azonenberg> right now i'm trying to get the basic ux for plot areas figured out
<d1b2> <johnsel> regardless, this project seems to have a lot of advanced functionality implemented that we can make use of
<azonenberg> yes
<d1b2> <johnsel> I would not be surprised if it has some fill window code too
<d1b2> <johnsel> because those windows look filled
<d1b2> <johnsel> nfd is the library I meant
<d1b2> <johnsel> nativefiledialog-extended
<d1b2> <johnsel> you might also make a fake imgui component which is just an invisible draggable holder for whatever you want to make draggable
<d1b2> <azonenberg> So here's a test with two waveform areas in the left and one in the right
<d1b2> <azonenberg> where a waveform areas is just an ImGui::Image object
<d1b2> <johnsel> I see you're trying to invoke the dark lord
<d1b2> <johnsel> did it work?
<d1b2> <azonenberg> (currently showing the font texture as we don't have any actual drawing shaders yet)
<azonenberg> right now most of the layout is hard coded and theres not much of the way in backing data structures. just fooling around with a mockup
<azonenberg> but my goal is to make it so i can drag these areas freely from group to group
<azonenberg> and then figure out a good UI for creating new groups
<d1b2> <johnsel> one thing that comes to mind is there are node editors made with imgui
<d1b2> <johnsel> which are sort of a proto-example of what you want
<azonenberg> Yes. i plan to use that for the filter graph editor
<azonenberg> this is a bit different
<azonenberg> Anyway, give me time to tinker and we'll see what i come up with
<d1b2> <johnsel> presumably the code in there will be generic enough to work for both their usecase and yours
<d1b2> <johnsel> yeah they have just a generic DragAction implemented for their custom widgets/context
<d1b2> <johnsel> I'd probably go from there since it also implements things like copy paste and those kinds of usual suspects
<azonenberg> well i will likely want to do custom dragging anyway to handle things like splitting
<azonenberg> i.e. if i drag a waveform to the side of a group, i want to split the group
<d1b2> <johnsel> yes this is also just custom code to extend imgui
<d1b2> <johnsel> I think no matter what you'll end up with some custom widgets and extensions anyway but that is fine and the framework expects that anyway
<azonenberg> yeagh
<azonenberg> yeah*
<azonenberg> buttons on top of the texture
<d1b2> <johnsel> yeah not a big deal right
<azonenberg> i will probably add a new widget to the right and make the y axis scale actually be its own widget
<d1b2> <johnsel> just has to stay correct when you go moving things around
<d1b2> <johnsel> and scaling them
<d1b2> <johnsel> but that's always a bitch
<azonenberg> Yeah. But so far things are going very smoothly
<d1b2> <johnsel> also keep a profiler near 😛
<_whitenotifier-7> [scopehal-apps] azonenberg pushed 2 commits to master [+0/-0/±4] https://github.com/glscopeclient/scopehal-apps/compare/e1b69f959ad9...85af0e870166
<_whitenotifier-7> [scopehal-apps] azonenberg a9c02c2 - Initial experiments on docked waveform groups
<_whitenotifier-7> [scopehal-apps] azonenberg 85af0e8 - Continuing to experiment with waveform area docking etc
<azonenberg> We're doing 60fps without breaking a sweat so far
<d1b2> <johnsel> you should disable vsync
<d1b2> <johnsel> so you get the actual framerate
<d1b2> <johnsel> I'd guesstimate 300-400fps if you did not do anything funky just yet
<azonenberg> oh easily
<azonenberg> but i'd rather just throw it in nsight graphics
<azonenberg> and measure actual render time etc
<d1b2> <johnsel> if you run it through nsight with vsync on that will absolutely lose a lot of precision in the data
<azonenberg> no, because i care about how long the draw calls take
<azonenberg> in any case as soon as we start doing heavy stuff aka drawing waveforms
<azonenberg> that will become the bottleneck
<azonenberg> this is nothing
<d1b2> <johnsel> you're influencing the way it will plan work with vsync on, it's absolutely not the way to measure the actual performance
<d1b2> <johnsel> you essentially tell it it does not matter how it plans the work, as long as it reaches 60fps, so it definitely prioritizes work differently based on that
<d1b2> <johnsel> and you arbitrarily limit the performance so you don't see your margin anymore
<azonenberg> yeah but i'm interested in how long individual commandbuffers take to execute
<azonenberg> not the overall elapsed time
<azonenberg> the only thing that blocks on vsync is the final swapchain present call, no?
<d1b2> <johnsel> sure, but those are also influenced by having vsync on
<d1b2> <johnsel> no, there definitely happens more under the hood than just the final swapchain present call
<d1b2> <johnsel> you can easily see if it you do a run with it off that there will be differences in execution order
<d1b2> <johnsel> and if not yet, at some point
<d1b2> <johnsel> in fact, nsight should force vsync off
<d1b2> <johnsel> at least according to this doc, but it is for d3d11 https://docs.nvidia.com/nsight-graphics/2018.4/content/nsight_graphics/range_profiler_d3d11.htm
<d1b2> <johnsel> there's also the CPU end of things, if there's a 18ms window to plan all the memory transfers or a 2ms window it will definitely be much easier to hide a problem in the first case
<d1b2> <johnsel> and all the governors and whatnot have this info when they make their decisions
<d1b2> <johnsel> yep latest docs say it on the getting started page too Why does my application run at a different frame rate under Nsight Graphics? The Nsight Graphics Frame Debugger disables VSYNC, so applications that have VSYNC enabled under normal circumstances may see a higher frame rate when the same application is run under the Frame Debugger. Nsight Graphics also has a small performance overhead, which may reduce the frame rate slightly.
<_whitenotifier-7> [scopehal-apps] azonenberg pushed 1 commit to master [+0/-0/±2] https://github.com/glscopeclient/scopehal-apps/compare/85af0e870166...7b623fbb3214
<_whitenotifier-7> [scopehal-apps] azonenberg 7b623fb - Fixed packing of waveform areas
<d1b2> <johnsel> yep, Note: For best accuracy, it is recommended that you run your application in full-screen mode and turn off VSYNC. You can turn VSYNC off from your application or set DXGI SyncInterval to 0 under Additional Options in the connection dialog.
<d1b2> <johnsel> I'm not making this up 🙂
<azonenberg> lol
<azonenberg> If i'm reading this right, you know how long a frame takes right now?
<azonenberg> 210 us :p
<azonenberg> or about 4700 FPS :p
<d1b2> <johnsel> send a picture lol I want to confirm that
<d1b2> <johnsel> that is a little faster than I expected, but it's definitely not in the realm of the impossible
<azonenberg> We would probably be CPU limited if we tried actually going that high
<azonenberg> this is just the GPU time
<d1b2> <johnsel> there's a good chance that no matter what you'll end up cpu limited
<azonenberg> Yeah
<d1b2> <johnsel> but that's irrelevant, you just want as fast as possible
<d1b2> <johnsel> cpu or io limited in fact
<azonenberg> well, the other thing is i had been cpu limited before under gtk glscopeclient
<azonenberg> so i'm aggressively pushing as much as i can to the gpu :p
<d1b2> <johnsel> yeah I understand
<d1b2> <johnsel> I have been following your twitter posts for some time
<d1b2> <johnsel> I've seen it happen hehe
<d1b2> <johnsel> are you sure that's the complete frame though?
<azonenberg> OK here we go
<azonenberg> /*vk::PresentModeKHR::eFifo*/vk::PresentModeKHR::eImmediate,
<azonenberg> we're bouncing betwen about 500 and 1000 FPS, averaging 700ish
<d1b2> <johnsel> that seems reasonable
<d1b2> <johnsel> now I wonder how it will perform on the Windows end
<d1b2> <johnsel> I think pretty similar
<d1b2> <johnsel> which would be amazing compared to what it was
<d1b2> <johnsel> ok you've convinced me let me try to build it
<azonenberg> just patch VulkanWindow.cpp:200
<azonenberg> with that mode change
<d1b2> <johnsel> do you have a recommendation for which commit to take?
<azonenberg> current git head
<d1b2> <johnsel> alright here I go
<d1b2> <johnsel> wait
<d1b2> <johnsel> do we even need mingw now
<azonenberg> i have no idea
<d1b2> <johnsel> how much dependencies does this pull in?
<d1b2> <johnsel> I'll look myself nvm
<azonenberg> i mean, it depends on scopehal and scopeprotocols
<azonenberg> whcih pull in a few gdk components at least for now
<azonenberg> they use gdkcolor's for example in scope channels
<azonenberg> we can change that
<d1b2> <johnsel> yeah.. hmmm
<azonenberg> and then we use sigcxx for callbacks when a filter property changes
<azonenberg> but that might well be possible to pull in with much less overhead than full gtk
<d1b2> <johnsel> libsigc++ is used by gtkmm to wrap the GTK signal system. It does not depend on GTK or gtkmm.
<d1b2> <johnsel> that would be fine
<d1b2> <johnsel> we can just make that a local dependency
<azonenberg> and yeah thats what i mean
<d1b2> <johnsel> or pull in a binary
<d1b2> <johnsel> whatever
<d1b2> <johnsel> it can stay
<azonenberg> and then there is a small gdk dependency but it would be easy to refactor away
<d1b2> <johnsel> but that does not solve my immediate issue right now
<azonenberg> That is something to work on once ngscopeclient reaches feature parity with glscopeclient IMO
<d1b2> <johnsel> can i just, not include scopehal?
<azonenberg> not a near term priority
<azonenberg> You could probably patch it out pretty easily
<azonenberg> the only thing i am doing in scopehal right now is connecting to the scope and running a background thraed
<azonenberg> to poll for trigger events
<azonenberg> you can easily remove that functionality if you just want to benchmark the gui
<d1b2> <johnsel> well the argument against that would be that if we don't have to drag all those dependencies in and have whatever build and CI complexities they bring we might become more productive sooner
<azonenberg> well we still need them for gtk glscopeclient
<azonenberg> which is not going away any time soon
<azonenberg> i expect end of year very optimistically is the soonest we could possibly get to the point of being able to completely replace it
<azonenberg> more likely some time in 2023
<d1b2> <johnsel> I guess that's a Monday discussion, but I do think that if we do go that route we'll keep having to backport fixes for Windows (and perhaps osx too) that don't really add to the product
<azonenberg> We may decide to table some of the porting work until ngscopeclient is deployable
<d1b2> <johnsel> we're basically sweeping together loose bits and pieces that fall off our dependencies
<azonenberg> That will be a decision that has to be made in parallel with the folks funding the osx porting of course
<azonenberg> but if we have a good argument to say the engineering time is better spent elsewhere, i think matt will be ok with it
<azonenberg> in any case the vulkan shader porting is unaffected
<azonenberg> that is on the critical path for both ngscopeclient and the osx port of glscopeclient
<d1b2> <johnsel> well I'm not being funded so I think that should just be seen as another stakeholder in the decision but from my end of things it seems stupid to build a better foundation but then first port in all the reasons we're moving to a better foundation only to drop them afterwards
<d1b2> <johnsel> does that make sense?
<azonenberg> yeah
<azonenberg> what i'm saying is , though, i am in no hurry to get ngscopeclient running on other platforms ASAP
<d1b2> <johnsel> but I realize I'm not the only stakeholder for sure, that's why I said it's a Monday discussion
<azonenberg> if it's held back by a library we currently use in scopehal that we can migrate away from, it's not a huge deal
<azonenberg> ngscopeclient is a R&D testbed as of now
<azonenberg> glscopeclient is the main project, and gtk etc are core dependencies there
<d1b2> <johnsel> that's a trap though, because you might invest a lot of time in a direction that for some esoteric reason might turn out problematic on one of the other platforms you want to support
<azonenberg> pretty much all of what i'm doing now is just imgui widget twiddling to get the ui structure i want
<azonenberg> that should be quite portable
<d1b2> <johnsel> sure, but that's precisely why I argue for porting it now, and keeping feature parity with other platforms and CI as soon as possible
<azonenberg> well, getting gtk glscopeclient working on all of the platforms at least somewhat is still important
<d1b2> <johnsel> automate a performance profiling step, so we see how things are affected
<d1b2> <johnsel> yeah sure and that should happen
<azonenberg> I dont see this as changing the near term direction of the project
<azonenberg> if gtk glscopeclient runs on everything
<azonenberg> ngscopeclient should too
<azonenberg> its dependencies as of now are a strict subset
<d1b2> <johnsel> well you can easily foresee exercising code paths with future code that you don't with the old code
<azonenberg> sure
<d1b2> <johnsel> and the whole CI could be done properly and lean now but not if we add in all those dependencies that we'd barely use
<azonenberg> i'm still not quite clear on what you're suggesting then
<azonenberg> we have to keep gtk and such around to build glscopeclient
<d1b2> <johnsel> I'm suggesting that we try to lose those dependencies in ngscopeclient
<azonenberg> are you suggesting a completly separate build tree then?
<azonenberg> so you can build ngscopeclient without installing gtk?
<d1b2> <johnsel> ideally/potentially yes
<azonenberg> So, let me think a bit
<azonenberg> libscopeexports is heavily GTK, but not a core component
<azonenberg> (it will be completely rewritten for ngscopeclient based on imgui or ideally a more data driven workflow)
<azonenberg> sigcxx is used throughout
<azonenberg> i think honestly, gdk colors are the main thing we still use in scopehal/scopeprotocols
<d1b2> <johnsel> that's a lot of GDK for little functionality
<azonenberg> Correct. i plan to lose it
<azonenberg> so, proposal: refactor scopehal to use std::string of hex codes or a float[] or some other representation tbd of colors
<d1b2> <johnsel> then (pending input from lain) I think we could do a very lean build for ngscopeclient
<azonenberg> at that point scopehal and scopeprotocols should be able to build sans gtk
<azonenberg> keep it in a single cmakelists
<azonenberg> but make gtk a configure time detection rather than a mandatory component
<azonenberg> and simply gate off a bunch of the binaries if it's not present
<azonenberg> i.e. build the libs and ngscopeclient only
<azonenberg> then we can simply delete the other subdirs once ngscopeclient reaches feature parity
<d1b2> <johnsel> I'd even go so far as putting ngscopeclient in a whole different repo
<d1b2> <johnsel> because we can also tag commits with version and use the release functionality of github
<azonenberg> I don't want to do that because i want to preserve history when we, for example, move/copy the preferencemanager over
<d1b2> <johnsel> if the scopehal repo is a submodule that history is implicitly there, no?
<azonenberg> no i mean scopehal-apps history
<azonenberg> or the shaders
<azonenberg> there are components in scopehal-apps we are going to repurpose
<d1b2> <johnsel> you can make that a submodule of ngscopehal too
<azonenberg> And i want that history preserved
<azonenberg> eeeeh that sounds like a huge pain for little gian
<d1b2> <johnsel> not on the CI end of things
<azonenberg> just make it so that you can build ngscopeclient without installing gtk
<d1b2> <johnsel> the alternative is that we would start making branches very strictly
<azonenberg> the fact that there's other components in the tree that we don't build shouldn't hurt anything
<azonenberg> its just like not using asan, or building the pdf docs
<d1b2> <johnsel> sure but how will you do your builds leanly if you don't know what you need to build
<d1b2> <johnsel> you can either build everything or nothing
<azonenberg> no, you just have a if(HAVE_GTK) include_directories(src/glscopeclient) etc
<d1b2> <johnsel> ideally you kick off a CI process only for the part of the code that has changed
<d1b2> <johnsel> so you get 10-20 minute builds instead of an hour
<azonenberg> ideally we'd have beefier builders so it's not 10 minutes either
<d1b2> <johnsel> sure, but that still won't happen without leaning up the builds
<azonenberg> I have no problem exploring funding for that
<d1b2> <johnsel> and if you just have that repo and 2 apps coming out of it you still won't know what code has changed and what to build
<d1b2> <johnsel> so you can only build everything, and that becomes expensive no matter what
<azonenberg> yeah so you do one mingw build and one native vc++ build
<azonenberg> and then we deprecate the mingw build when we drop the gtk side completely
<azonenberg> i have no problem with slow builds for a couple of months until we transition completely
<azonenberg> Look to the future but keep the status quo until the transition is complete
<azonenberg> also, for reference re hour long builds vs good hardware
<azonenberg> i just did a full build after a "make clean"
<azonenberg> real 195.70 user 3686.01 sys 630.88
<azonenberg> so just over three minutes
<d1b2> <johnsel> I mean we can do it this way (monorepo) but my experience is that you will get cross contamination where accidentally code from older versions of glscopeclient gets replaced because that was in the branch for ngscopeclient that you were working on (or somebody else rather)
<d1b2> <johnsel> and you lose the ability to do github releases
<azonenberg> it's not a branch, it's a subdir. they're separate binaries
<azonenberg> we will never do a github release during the crossover period
<d1b2> <johnsel> yes but you will work on them in separate branches
<azonenberg> at this point my intention is to push hard and get feature parity as quickly as possible then cut ties with the past
<d1b2> <johnsel> so somebody will check out code from some HEAD and work on it, that branch diverges into some other direction, and when you merge it back up you have to be very careful that it will not revert code because it thinks they are legitimate changes
<azonenberg> like i said, they're separate dirs
<d1b2> <johnsel> git does not know that
<d1b2> <johnsel> do you work only in master?
<azonenberg> what you're suggesting is as silly as saying that a change made to scopehal will magically appear in scopeprotocols
<azonenberg> 99% of the time, yes. I made a feature branch for the data model refactoring because i wanted to do intermediate commits that didn't compile, and only merge to master after the refactor was done
<azonenberg> but i still consider the whole codebase to be a prerelease alpha state where rapid changes are to be expected
<d1b2> <johnsel> then I understand why you don't see the issue because you'd never run into it
<azonenberg> there is no stable branch vs dev branch
<d1b2> <johnsel> once you start using feature branches more heavily at some point git will get confused and revert code because it doesn't detect that it's not a change
<azonenberg> i still dont understand what you're talking about
<azonenberg> branch or master doesnt matter, i'm not changing other files. they're separate projects
<azonenberg> also, i expect that the rate of change to glscopeclient is going to drastically slow down over the coming weeks
<azonenberg> because i'm ~the only dev on that side
<d1b2> <johnsel> I don't think I can explain it better but it's just some behavior of git where after enough changes and branching away from main it will confuse old code for new code
<azonenberg> so if i'm not touching it, nobody is
<azonenberg> therefore it will, for the most part, just sit there until we transition away
<azonenberg> people can use it, we'll fix bugs if i find them, but there won't be any major changes other than lain's shader work
<d1b2> <johnsel> yeah well if we ever run into it I'll point out the issue but if you plan to leave things mostly as is then it's not as big of a deal
<azonenberg> if you go back far enough
<azonenberg> this is what we had :p
<azonenberg> (this was before we had the top level build system and submodules in this repo so there's no scopehal submodule yet)
<azonenberg> i started glscopeclient a week or two after that commit
<azonenberg> developed in parallel, then transitioned over completely
<azonenberg> So the plan is to do exactly the same thing
<azonenberg> And the history traces that far back (prior to this, it was in subversion iirc)
<azonenberg> i dont think i bothered to replay the history through that
<d1b2> <johnsel> that's a progressed position from earlier, my experience with monorepos that have diverging code paths in different branches is just that it will cause some problems
<d1b2> <johnsel> but if that is not the expected thing to happen then it will be fine
<azonenberg> Yeah. For the most part the dev flow has been very linear
<azonenberg> we will likely start branching more heavily once we get to the point of having stable releases people are using
<azonenberg> and perhaps backporting fixes for bugs to those while doing new dev
<azonenberg> but that is still quite a way out
<d1b2> <johnsel> well at that point you will find out what I mean when you suddenly have to fix things twice on the same branch ^_^
<d1b2> <johnsel> but alright let's recap what we will do right now
<d1b2> <johnsel> I'd like to build ngscopeclient on Windows
<d1b2> <johnsel> I'll just comment out the dependencies for now
<d1b2> <johnsel> and then you will focus on removing the GTK dependency in scopehal etc
<d1b2> <johnsel> and I need to get libsigc++ built as an independent dependency on Windows
<d1b2> <johnsel> and then we can set up a CI job that does not go through the entire MinGW disaster
<d1b2> <johnsel> but that does mean we need to build using something else which I have to figure out what
<d1b2> <johnsel> probably Ninja directly from the CMakeLists
<d1b2> <johnsel> am I forgetting anything?
<d1b2> <johnsel> mm brb food
<azonenberg> Sounds like a plan
<d1b2> <xabean> @johnsel are .. are you talking about where a git commit can have multiple parents, and it could get confused in a "I am my own grandpa" cyclic situation?
<d1b2> <johnsel> No it's just a really complicated version of a merge conflict that arises from having multiple diverging codebases in different branches that converge and diverge a few times at different intervals. At some point you'll backport a fix that gets reverted back in a later commit because another branch is merged back in that does not have the fix and it does not correctly pick up which version is the correct one. Or because it has the change in
<d1b2> some other part of the history. It's a fairly common regression issue that large monorepos get (especially if the people git-ing are not very comfortable with it).
<d1b2> <johnsel> http://www.linux-kongress.org/2009/slides/fighting_regressions_with_git_bisect_christian_couder.pdf I think this explains it somewhat well and also offers a method to manage the problem
<d1b2> <johnsel> probably start reading at p53 or so
<d1b2> <johnsel> maybe a bit earlier
<d1b2> <johnsel> nonetheless the git bisect method is an interesting technique for everyone to know
<d1b2> <johnsel> to locate which commit introduces a problem
<d1b2> <johnsel> by using bsearch to locate it quickly based on having some good commit and some bad commit
<d1b2> <johnsel> anyway with the plan to keep a mostly linear git history I don't think we'll run into it too much
<d1b2> <xabean> it's a good read, thanks
<d1b2> <xabean> yeah this situation sounds a lot like it arises when the DAG gets really convoluted
<d1b2> <johnsel> convoluted, or just be diverged for a long time is already enough
<azonenberg> Yeah. So it's a total non-issue in our case where we generally work off of master and PRs get merged fairly quickly in most cases
<azonenberg> Also, re probes: just checked tracking, those 20 ohm 0201s should be here tomorrow
<azonenberg> So then i can test if this fixes my ADL5569 input match issue on the AKL-AD4
<d1b2> <johnsel> I wouldn't say it's a non-issue but there's a cost to changing involved too that I'm not convinced works in our favor
<d1b2> <azonenberg> @Makerprobe re our previous discussion: here's the response of the AKL-AD4 right now (including cable losses) from DC to 8.5 GHz
<d1b2> <azonenberg> My conjecture is that the ripple is due to a reflection between the amplifier and the tip resistor from the amplifier input not being matched to 50 ohms
<d1b2> <johnsel> anyway looking at the ngscopeclient build, in an ideal world it would be possible to just use vcpkg (which is a Windows package manager extension to CMake) to manage the packages so that we can just do cmake --build . to kick off the build
<d1b2> <azonenberg> Seen here, 1.25 Gbps eye pattern. seems like we have two little humps then it stabilizes
<azonenberg> johnsel: Well, look into it. I'll add a few items from our discussion today to the agenda for monday's meeting
<azonenberg> johnsel: also, what are your thoughts on cairo in all of this
<d1b2> <johnsel> well, one thought is that it is in vcpkg
<d1b2> <johnsel> I honestly am not familiar enough with it to have an opinion really
<d1b2> <johnsel> I'd say it's probably possible to keep but it depends on how hard you need it and the performance impact if I would like it
<d1b2> <johnsel> but in principle I am not against it
<azonenberg> i dont think there is any cairo dependency in scopehal but glscopeclient currently uses it for software rendered overlays and things like cursors, protocol decode overlays, etc
<azonenberg> and the timeline, axis labels, etc
<azonenberg> so basically the question boils down to, exactly how much of that code do we want to rewrite in imgui / custom vulkan rendering
<d1b2> <Makerprobe> Might be. Do you have a spice model of the probe?
<d1b2> <johnsel> I'm not sure how much effort there is to rewrite and what the downside is to keeping it
<azonenberg> vs keeping the current cpu-side renderer and shoveling a texture into things
<azonenberg> makerprobe: Most of my simulation to date has been Sonnet EM solver models of the PCB combined with a vendor s-parameter model of the amplifier
<azonenberg> and my own characterization data of the tip resistor
<d1b2> <johnsel> it's not a very modern library that's for sure
<azonenberg> although i'm not 100% thrilled with the quality of the de-embed there. I want to build a better test fixture with a 2x thru standard so i can de-embed vna measurements of the resistor
<azonenberg> anyway, my current sims actually show a worse reflection than i am observing in the VNA data
<azonenberg> but adding input resistance seems to improve it in sim
<azonenberg> So i figured it would be worth trying in hardware
<azonenberg> for reference, the AKL-AD4 is essentially two 500 ohm transmission line probes feeding into a differential amplifier
<d1b2> <johnsel> re: cairo I think that we should define the effort to move away from it and the performance impact and then decide
<azonenberg> so you get 1K ohm differential / 250 ohm common mode impedance as seen by the DUT
<d1b2> <johnsel> as far as the impact on the builds it seems much less problematic than the whole GTK stuff, which was my big issue with it besides the performance
<azonenberg> But the amplifier input dips to like 30 ohm impedance at a couple of GHz according to the vendor .s2p
<azonenberg> .s4p*
<d1b2> <Makerprobe> Well I can't really comment on that, my world was a lot simpler until now
<azonenberg> Lol. yeah i've been neck deep in microwave land for a while on these probes
<d1b2> <azonenberg> Here (blue curve, red is a previous PCB revision) is the AKL-AV1, the high impedance active probe i was talking to you about earlier
<azonenberg> it's a 5:1 R-C divider followed by a BUF802
<d1b2> <johnsel> BUF802 datasheet (or AN) has some notes on higher frequency behavior
<d1b2> <Makerprobe> To give you (and maybe the rest) a bit more background: I'm a mechanical engineer, my day job is district heating hydraulics. I debug those systems on a city level. Everything I do in electronics is really just my hobby for now.
<azonenberg> i have a new PCB rev that makes a few small changes to the power supply as well as making the bodges from the blue trace permanent (what you see there is the result of reworking the red board which is the actual PCB as-designed)
<azonenberg> waiting on a solder stencil and i can populate and test
<azonenberg> And I see. Well, this will be a great place for you to get up to speed and learn more
<azonenberg> The community of people designing their own probes is very small and a good number of them are hanging out here
<d1b2> <azonenberg> AKL-AV1 on a random wireless router board as a test subject
<azonenberg> (and my degree is in computer science, I learned all of this on my own)
<d1b2> <Makerprobe> And that's great! Until now my projects were driven by my own needs and those of people who I found and who somewhen found me
<azonenberg> basic architecture: 100 ohm damping resistor from tip to DUT, then R-C divider. 4M resistor on back side of PCB, 250-750 fF trimmer (white circle) on front of board
<azonenberg> BUF802 in composite loop mode with a precision opamp on the back of the board is the main amplification stage
<d1b2> <Makerprobe> I tried to avoid parts on the back of my fet probe because I wanted to make it glue-able
<azonenberg> trimmer bottom right lets you trim the low frequency gain to match high freq, then the trim capacitor is for matching the R-C divider to match the ~2.4 pF input capacitance of the BUF802
<azonenberg> then the rest of the parts on the right side are all power regulation
<azonenberg> the new redesigned version reduces it from 3 ICs to 2
<azonenberg> (I haven't put EM shield cans on any of my designs yet, but likely will try that at some point)
<azonenberg> and yeah, mine were optimized for small size. Re glueable, you mean gluing the board to the DUT or what?
<d1b2> <Makerprobe> Yes
<azonenberg> This is the moutning scheme I came up with
<azonenberg> solid copper wire soldered to a pad on the underside of the probe head, then that is taped to the DUT w/ double sided adhesive
<d1b2> <Makerprobe> Looks smart
<azonenberg> you can bend it to pose the probe head at a point of your choice
<azonenberg> I have four different probes now using the same foot design
<d1b2> <Makerprobe> That's what I thought
<azonenberg> (note that the foot is electrically isolated, not on probe ground, to avoid coupling noise into the probe)
<azonenberg> it's soldered purely as a mechanical attachment method
<azonenberg> which, while relatively permanent, still allows easy replacement if the wire fatigues and snaps
<azonenberg> or customization if the end user prefers a different length or stiffness of mount
<d1b2> <Makerprobe> It's remarkable that we have absolutely different needs electrically, but the mechanical challenges are the same
<d1b2> <Makerprobe> My probes have to cover higher voltages, but lower bandwidth. Still need to be mounted somehow
<azonenberg> Yep. fundamentally the probe still has to attach to the dut, be able to hold itself at the right spot, avoid transferring excessive forces to the tip
<azonenberg> etc
<azonenberg> Roughly speaking the AKL-AV1 is positioned as a solder in competitor to the likes of https://teledynelecroy.com/probes/active-voltage-probes/zs1500
<azonenberg> The AV1 is as far as i can tell superior to the ZS1500 in every regard. The AD4 has higher loading than the name brand diff probes, but for a lot of applications 1K ohm zdiff is totally fine
<azonenberg> And then there's the single ended, passive PT5
<azonenberg> seen at left, with my 13 GHz lecroy differential at right for comparison
<azonenberg> you can see they have a similar mounting architecture, but it's a removeable clip while mine is semi-permanently affixed
<azonenberg> (the IC on there is just a filter, there's no power source or gain)
<d1b2> <Makerprobe> That MMCX. does it rattle? I think I use the same connector on some of my pcbs
<azonenberg> That one is almost ready to scale up to production, I'm just making some final tweaks for flatness
<azonenberg> It's not a MMCX, it's SMPM
<azonenberg> MMCX tops out at 6 GHz :)
<azonenberg> very similar dimensionally but designed for higher frequency, i think it's rated for like 40 GHz or so
<d1b2> <Makerprobe> Ah right
<azonenberg> I use SMPM on my lower BW probes too for parts commonality
<d1b2> <Makerprobe> See, my world ends far below that
<azonenberg> Yeah. Like i said we're targeting different parts of the design space
<d1b2> <azonenberg> pair of PT5s looking at a PCIe to gigabit fiber ethernet network card
<d1b2> <azonenberg> one on the pcie side and one on the ethernet side
<azonenberg> i got some really cool captures that i did protocol decoding of in glscopeclient, you can see the pcie TLPs going in and the ethernet frame coming out, and measure the latency through the card and everything
<azonenberg> and decode the protocol on both sides
<azonenberg> (this was actually the older 5.3 GHz version of the PT5 in this photo I think, I've since pushed it a lot further
<d1b2> <azonenberg> yeah you can see here there is a tip resistor and then an 0201 on the PCB right behind it
<azonenberg> You get reflections between the two which limits BW. I had this design because at the time I started R&D, I could only get this resistor in 100 ohms which was too small to use by itself
<azonenberg> I ordered some 450 ohm but it took like 6 months to ship
<azonenberg> makerprobe: are you looking at handheld designs too?
<azonenberg> i have several ideas for handheld probes I want to explore as well that i'm not currently pursuing. one in particular i have never seen done
<azonenberg> a conventional 10M ohm R-C divider probe with two pins for signal and ground at adjustable spacing with some kind of screw system for changing the pitch
<azonenberg> like how high end differential probes work
<azonenberg> i've never seen a passive probe built like that, but there's no reason why it can't be done
<d1b2> <Makerprobe> yes! actually most of my pipeline is handheld stuff
<azonenberg> ah i see
<d1b2> <Makerprobe> spacing cound be adjusted by excentric pogos (for low-ish bandwidth)
<d1b2> <Makerprobe> let me look for a picture
<azonenberg> i also have a handheld transmission line probe, the AKL-PT1, that i'm fighting flatness issues (peaking around 5 GHz) on
<d1b2> <Makerprobe> of the handheld stuff
<azonenberg> and i want to make a handheld differential probe. that one also got shelved because of flatness problems. it actually had a big null in the response caused by the amplifier input not being well matched and causing reflections (gee sounds familiar lol)
<azonenberg> same amplifier chip
<azonenberg> i was hoping putting it right by the tip would improve response
<azonenberg> And it did, now i have a ~3 dB null at 4 GHz instead of like a 10 dB null at 1 GHz lol
<azonenberg> but it's still not where i want it :p
<azonenberg> anyway, if you're interested in collaborating at some point, I definitely need mechanical engineering work done on my handheld designs eventually :)
<azonenberg> they're just not where they need to be electrically yet
<azonenberg> once i have them working well we can focus on tuning ergonomics and such
<d1b2> <Makerprobe> this would be an enclsoure for a handheld diff probe we're developing. 50 V CM upwards
<d1b2> <Makerprobe> It's got a tip light!
<d1b2> <Makerprobe> power supply via a 4 pin 3.5 mm audio jack, output MMCX
<d1b2> <Makerprobe> the same enclosure would work for a single ended probe (DC coupled) and an AC coupled FET input probe similar to what I posted earlier (but, well, hendheld)
<azonenberg> Nice. so far all of my active probes are powered via a single DC supply on a U.FL because they're so size constrained
<azonenberg> and any negative rails i might need are generated on board
<azonenberg> i found a very nice combo charge pump and LDO from TI, iirc LM27761, that i've been using
<azonenberg> replacing several previous options
<azonenberg> and yeah my diff probes are like 5V max common mode or something lol
<azonenberg> if that
<d1b2> <Makerprobe> we tried specialized chips as well but opted for a more generic approach
<azonenberg> That's one of the things you usually give up when you are aiming at GHz bandwidth
<azonenberg> for my higher BW probes i might actually have to split bands
<azonenberg> have several different amplifiers and diplexers and then recombine on the other side
<azonenberg> I have not actually attempted such a design yet, but i don't think e.g. DC to 16 GHz will be possible with a single chip solution
<d1b2> <Makerprobe> I need to go to bed...almost midnight
<azonenberg> even DC to 8ish GHz is hard to do that way
<azonenberg> and ok enjoy :) see you around
<d1b2> <Makerprobe> see you
<d1b2> <johnsel> PS C:\GitHub\scopehal-apps\src\ngscopeclient> cmake --build . MSBuild version 17.3.1+2badb37d1 for .NET Framework Checking Build System Building Custom Rule C:/GitHub/scopehal-apps/src/ngscopeclient/CMakeLists.txt imgui_impl_vulkan.cpp C:\GitHub\scopehal-apps\src\imgui\backends\imgui_impl_vulkan.h(46,10): fatal error C1083: Cannot open include file: 'vulkan/vulkan.h': No such file or directory
<d1b2> [C:\GitHub\scopehal-apps\src\ngscopeclient\ngscopeclient.vcxproj] MainWindow.cpp C:\GitHub\scopehal-apps\src\ngscopeclient\ngscopeclient.h(32,10): fatal error C1083: Cannot open include file: '../scopehal/scopehal.h': No such file or directory [C:\GitHub\scopehal-apps\src\ngscopeclient\ngscopeclient.vcxproj] VulkanWindow.cpp C:\GitHub\scopehal-apps\src\ngscopeclient\ngscopeclient.h(32,10): fatal error C1083: Cannot open include file:
<d1b2> '../scopehal/scopehal.h': No such file or directory [C:\GitHub\scopehal-apps\src\ngscopeclient\ngscopeclient.vcxproj] main.cpp C:\GitHub\scopehal-apps\src\ngscopeclient\ngscopeclient.h(32,10): fatal error C1083: Cannot open include file: '../scopehal/scopehal.h': No such file or directory [C:\GitHub\scopehal-apps\src\ngscopeclient\ngscopeclient.vcxproj] Generating Code...
<d1b2> <johnsel> not bad!
<d1b2> <johnsel> though I'm sure there's something to be said for the mess I made with the cmake config
<d1b2> <johnsel> but that's for later
<azonenberg> well let me know if/when you get it compiling that way
<azonenberg> i've always wanted to ditch mingw and get a native windows build
<d1b2> <johnsel> PS C:\GitHub\scopehal-apps\src\ngscopeclient> cmake --build . MSBuild version 17.3.1+2badb37d1 for .NET Framework MainWindow.cpp C:\GitHub\scopehal-apps\src\ngscopeclient\VulkanWindow.h(44,26): error C2039: 'string': is not a member of 'std' [C:\GitHub\scopehal-apps\src\ngscopeclient\ngscopeclient.vcxproj]
<azonenberg> yeah scopehal.h includes <string>
<d1b2> <johnsel> I am only building ngscopeclient
<azonenberg> so if you removed scopehal then you'll probably have to patch in a bunch of STL headers
<d1b2> <johnsel> I'll probably run into a few of those right
<d1b2> <johnsel> are they by any chance nicely together?
<azonenberg> its probably just string, vector, map
<azonenberg> you'll also need to patch out most of the code in the Session class that handles m_threads and m_oscilloscopes
<azonenberg> AddScopeDialog::DoConnect() can just return true
<azonenberg> and ScopeThread can be commented out to return immediately
<azonenberg> that's all groundwork for when we actually do useful things with instruments
<azonenberg> but at this stage none of it does anything of consequence
<d1b2> <johnsel> thanks, I think I'll get it with that
<azonenberg> (and those all depend on scopehal)
<d1b2> <johnsel> I look forward to hearing what you think of it (and vcpkg)
<d1b2> <johnsel> but I think it will work
<azonenberg> i dont do windows, it's always been someone else's problem
<azonenberg> so i have no opinion on the build process there
<d1b2> <johnsel> yes but since it hooks into cmake
<azonenberg> ah yeah
<d1b2> <johnsel> we'll have to see how we can either keep separate build processes or maybe merge them
<d1b2> <johnsel> but that's all dependent on it working first
<azonenberg> like i said, my thought to start was to just do a if(HAVE_GTK) gate on all of the other components
<d1b2> <johnsel> I do know it can do package management for linux
<d1b2> <johnsel> but we'll see
<azonenberg> Great. well this monday call should be quite the busy one lol
<azonenberg> we'll have a lot to talk about
<d1b2> <johnsel> indeed
<azonenberg> also speaking of which... hi mughees
<d1b2> <johnsel> well at least we won't have to get bored anytime soon
<d1b2> <azonenberg> oops discord didnt ping him, hi @Mughees
<azonenberg> he's a new dev from the same company louis is at, sounds like they have him working on an IIR filter block to complement our FIR
<azonenberg> hopefully he'll be joining the call to properly introduce himself to the team and let us know how things are going
<d1b2> <johnsel> yes I also need to properly introduce myself
<d1b2> <Mughees> Hi Guys, Im excited to join this effort…. For now I am getting used to the project.
<d1b2> <Mughees> Today I hot segmentation fault 😝
<d1b2> <Mughees> got*
<azonenberg> Lol. In your new code or just trying to run the app at all?
<d1b2> <Mughees> No i just took fresh master today
<azonenberg> get me a backtrace and let me know where it's crashing
<d1b2> <Mughees> Two weeks back i could run the app fine and play around
<d1b2> <Mughees> Probably my environment
<azonenberg> if it's deep in X11 libraries from GLFW
<azonenberg> then it's a crash that i've been chasing on one of my three machines that i can't reproduce on the other two
<azonenberg> i plan to add a workaround tonight for that
<d1b2> <johnsel> hmmmm, vcpkg_installed\x64-windows\include\sigc++-3.0\sigc++/tuple-utils/tuple_start.h(25,1): error C2429: language feature 'nested-namespace-definition' requires compiler flag '/std:c++17'
<azonenberg> but i don't have a root cause on it yet
<d1b2> <johnsel> do I care azonenberg?
<d1b2> <johnsel> or just go c++17 for now
<d1b2> <johnsel> 2.0 is not in the vcpkg repo at the point I'm at
<azonenberg> johnsel: We currently compile under c++ 14 because there's been no reason to bump to 17, and one of our compilers - i think actually the mingw one lol - did not support it at some point
<azonenberg> i know c++ 20 is off the table for a year or two because i dont think the debian gcc supports it yet
<azonenberg> so until debian bookworm comes out no 20
<d1b2> <johnsel> well this might be a reason to go to 17
<d1b2> <Mughees> Sure, first thing tomorrow
<azonenberg> but i have no objection to bumping to 17 at least on the windows build
<d1b2> <johnsel> I'll just go that route because putting other packages locally is a bridge to far for me for a test build
<azonenberg> basically, i've bumped when i had a reason to and all of our target platforms' compilers supported it
<d1b2> <johnsel> yeah we can review
<d1b2> <johnsel> it's not a dealbreaker so I can just temporarily do it now for this build
<azonenberg> Yep
<azonenberg> mughees: so over the past week or so i've been doing a lot of work on vulkan initialization stuff. i'm seeing some of it crashing on some machines and i dont yet know why.
<azonenberg> but the GUI side vulkan code is not yet used in glscopeclient anyway, only in the new experimental ngscopeclient frontend
<azonenberg> As a workaround, try commenting out all of the GLFW calls in scopehal/VulkanInit.cpp
<d1b2> <johnsel> might make sense to enable those debug layers that should help with these kinds of issues
<azonenberg> johnsel: I did. didnt help
<d1b2> <johnsel> they should help with pointing out why it crashes, no?
<azonenberg> it's dying deep inside xlib trying to do things like set cursors
<azonenberg> some state somewhere is being corrupted and i havent been able to root cause it
<azonenberg> GLFW initializing is a causative factor
<azonenberg> if we remove the glfwInit() call it stops dying
<azonenberg> also, it behaves differently in release, debug, and asan builds
<d1b2> <johnsel> it might be one of those weird initialise a few different GPU frameworks at the same time and have weird crashes that I have been preaching about
<azonenberg> very possible. except i also see the same crash in ngscopeclient
<azonenberg> on that box
<azonenberg> which afaik is not pulling in anything else
<d1b2> <johnsel> it's hard to say without having the actual trace
<azonenberg> This is one of several i'm looking at
<azonenberg> that's when i build in debug mode
<azonenberg> When i run under asan, it does not crash or give any memory errors, but it fails to find any queue types to render to
<d1b2> <johnsel> it does refer to glcore
<azonenberg> in particular, glfwGetPhysicalDevicePresentationSupport() reports that vulkan cannot render to queue type 0 - which i know it actually can
<azonenberg> this suggest something deep inside xlib is already corrupted by this point
<azonenberg> and if i comment out that call so it gets a bit further...
<d1b2> <johnsel> yeah it does smell like one one of those types of issues I meant
<d1b2> <johnsel> I don't understand why Vulkan init would end up in nvidia-glcore
<d1b2> <johnsel> but for some reason it thinks there's no device to render to
<d1b2> <johnsel> it might be the switch between GL having control over the device and Vulkan having control over the device
<d1b2> <johnsel> fighting over who gets to talk to the cursor
<azonenberg> nice theory: except that this also happens in ngscopeclient which has no gtk anywhere
<azonenberg> also VulkanInit() is called before i create any gtk windows or even x windows at all
<azonenberg> so there's no GL anywhere
<azonenberg> it's one of the first things i do in main() after argument parsing
<d1b2> <johnsel> it might be soemthing happening when you load a library like glfw
<d1b2> <johnsel> have you tried taking time out of the equation?
<azonenberg> ??
<d1b2> <johnsel> just wait a few ms
<azonenberg> it's possible glfw does some funky stuff. anyway, whatever it is is (so far) limited to my one machine
<azonenberg> wait where
<azonenberg> after initializing glfw?
<d1b2> <johnsel> before your problematic code
<d1b2> <johnsel> yes
<d1b2> <johnsel> see if you give it time to do whatever it might be doing it stops crashing
<azonenberg> lets find out
<d1b2> <johnsel> you say this happens before you create a window?
<azonenberg> yes
<azonenberg> and again it's different behavior with/without asan
<azonenberg> that screams memory corruption
<azonenberg> but corruption withi na structure field or something
<azonenberg> so it doesnt hti any red zones and throw memcheck errors
<d1b2> <johnsel> yes but fairly deep in the system at the X or driver level even
<azonenberg> exactly
<azonenberg> i tried the most recent nvidia driver as well as the same one i have been using on my other systems
<azonenberg> no change
<d1b2> <johnsel> which is why these things are so annoying, so many weird legacy code that you may be happy if something runs with just Vulkan or just GL or whatever
<azonenberg> it affects ngscopeclient
<d1b2> <johnsel> lots of weird 'this should never happen' behavior
<azonenberg> but not vulkan cube
<d1b2> <johnsel> let me read the code
<d1b2> <johnsel> I'll probably regret changing my focus once I am fully distracted but alright
<d1b2> <johnsel> this happens at #99 right?
<d1b2> <johnsel> just VulkanInit()
<d1b2> <azonenberg> it's in the loop in VulkanInit() after we choose a device to use
<d1b2> <azonenberg> and are checking to see what queue families are best
<d1b2> <johnsel> I'm guessing here but maybe there is some thread safety issue?
<d1b2> <johnsel> also Object sharing is implemented by the operating system and graphics driver. On platforms where it is possible to choose which types of objects are shared, GLFW requests that all types are shared.
<d1b2> <johnsel> perhaps it's cleaning up after some function call where it shouldn't
<azonenberg> the other possibility is that i am using the c++ vulkan API
<azonenberg> and creating my own instance
<azonenberg> while glfw is doing so as well
<azonenberg> so it's possible the two handles are stepping on each other somehow
<azonenberg> but again we're using the same nvidia driver version
<azonenberg> it shouldn't...
<d1b2> <johnsel> Request as many compute queues as we're allowed to, and make them all equal priority
<azonenberg> azonenberg@firefly:/ceph/fast/home/azonenberg/code/scopehal-apps/asan-build$ ldd ./src/glscopeclient/glscopeclient | grep vulkan
<azonenberg> libvulkan.so.1 => /opt/picovna/libvulkan.so.1 (0x00007fed51514000)
<azonenberg> well *that* explains a lot
<azonenberg> its not using the system libvulkan
* azonenberg investigates
<d1b2> <johnsel> that's... weird
<d1b2> <johnsel> but does it make a difference/
<d1b2> <johnsel> anyway it might be the queues
<azonenberg> why?
<d1b2> <johnsel> if you take up all the queues and glfw tries to make it's own queue
<azonenberg> no
<azonenberg> you can have multiple queues use the same ID
<azonenberg> they just share the same priority
<d1b2> <johnsel> that's not what I meant
<d1b2> <johnsel> if you have a limited amount of queues you may create
<d1b2> <johnsel> and you make them all
<d1b2> <johnsel> the next queue can't be made
<azonenberg> no
<azonenberg> that is not the case
<azonenberg> you can use the same queue index for multiple vkqueue handles
<azonenberg> i found this out by accident when i realized all of my threads were actually using queue 0
<d1b2> <johnsel> that might still invalidate the old reference
<azonenberg> nope, it actually returns the same handle
<d1b2> <johnsel> hmmm, I guess not then
<azonenberg> no i'm strongly suspecting the second copy of libvulkan
<azonenberg> just need to make sure i'm not using it, my first attempt it was still resolving there
<azonenberg> Confirmed
<azonenberg> at this point i'm 99% sure i understasnd
<azonenberg> root cause: glfw dynamically loads libvulkan if available on the target system (so that it can use opengl and vulkan in the same binaries even if you dont have vulkan installed)
<azonenberg> and because vulkan has a bunch of dynamic extensions etc anyway
<azonenberg> i was compile time linking to libvulkan in my app
<azonenberg> this is fine, if both are loading the *same* libvulkan
<azonenberg> but symbol resolution got borked and we ended up with two different libvulkans with incompatible state
<d1b2> <johnsel> seems you were spot on the first time then, nice intuition
<azonenberg> oh it was not the first time
<azonenberg> i spnet a long time chasing other oiptions
<d1b2> <johnsel> hehe
<d1b2> <johnsel> well
<d1b2> <johnsel> C:\GitHub\scopehal-apps\src\ngscopeclient\vcpkg_installed\x64-windows\include\vkfft/vkFFT.h(38,10): fatal error C1083: Cannot open include file: 'glslang_c_interface.h': No such file or directory [C:\GitHub\scopehal-apps\src\ngscopeclient\ngscopeclient.vcxproj]
<d1b2> <johnsel> we're at breakure parity
<azonenberg> (bvernoux opened a ticket against the SDK for that, i think the windows vulkan SDK doesnt include libglslang but the linux version does)
<d1b2> <johnsel> breaker parity?
<d1b2> <johnsel> yes I solved it a few days ago already
<azonenberg> and yeah i spent hours last night rebooting, installign different nvidia driver versions
<azonenberg> even going so far as to verify hashes of all of my gtk and x11 libs against the .deb packages
<azonenberg> in case of disk corruption or something
<d1b2> <johnsel> well you chased it down eventually
<d1b2> <johnsel> interesting
<d1b2> <johnsel> #ifdef APPLE #include <OpenCL/opencl.h>
<d1b2> <johnsel> vkfft
<d1b2> <johnsel> .h
<azonenberg> vkfft supports a lot of different backends
<azonenberg> opencl, cuda, hip, level0, and vulkan
<d1b2> <johnsel> yes but that's aside from the backend you choose
<azonenberg> huh it shouldnt include that unless the opencl backend is selected, but w/e
<d1b2> <johnsel> that's why I shared it
<d1b2> <johnsel> in case you/lain run into it
<azonenberg> anyway, ngscopeclient now runs on my box. who knows if i broke the picovna software
<azonenberg> i'll find out soon i guess :p
<d1b2> <johnsel> doth I need scopeexports?
<d1b2> <johnsel> no right?
<azonenberg> No
<azonenberg> ngscopeclient doesnt even link to it
<azonenberg> thats a bunch of gtk export wizards for saving waveform data to csv, touchstone, etc
<azonenberg> it needs a complete rewrite for imgui
<d1b2> <johnsel> it includes it
<d1b2> <johnsel> or it did at least lol
<azonenberg> yeah i just copied a lot of the build config from glscopeclient
<azonenberg> you might have to comment out the scopexportstaticinit() call in main
<d1b2> <johnsel> aw now I'm getting to the stupid part
<d1b2> <johnsel> C:\GitHub\scopehal-apps\src\ngscopeclient../../lib/log/log.h(56,17): error C2144: syntax error: 'unsigned int' should be preceded by ';' [C:\GitHub\scopehal-apps\src\ngscopeclient\ngscopeclient.vcxproj]
<d1b2> <johnsel> #include <mingw.mutex.h>
<d1b2> <johnsel> extern __thread unsigned int g_logIndentLevel; so ^that won't go
<azonenberg> __declspec((thread)) or so
<azonenberg> there is a thread local storage specifier on windows
<azonenberg> i forget exactly what
<azonenberg> i think you're the first person to try compiling this on vc++, rather than gcc on windows
<d1b2> <johnsel> I'm a rebel, what can I say
<azonenberg> lol
<azonenberg> hey, someone had to try
<d1b2> <johnsel> it's going quite well actually
<d1b2> <johnsel> oh no abort
<d1b2> <johnsel> -- Using cached msys-libreadline-8.1.001-1-x86_64.pkg.tar.zst. -- Using cached msys-ncurses-6.2-2-x86_64.pkg.tar.zst. -- Using cached msys-automake1.16-1.16.3-3-any.pkg.tar.zst. -- Using cached msys-perl-5.32.1-2-x86_64.pkg.tar.zst.
<d1b2> <johnsel> cairomm
<d1b2> <johnsel> grr
Johnsel has joined #scopehal