azonenberg changed the topic of #scopehal to: libscopehal, libscopeprotocols, and glscopeclient development and testing | https://github.com/glscopeclient/scopehal-apps | Logs: https://libera.irclog.whitequark.org/scopehal
<_whitenotifier-9> [scopehal-apps] azonenberg pushed 1 commit to master [+0/-0/±4] https://github.com/glscopeclient/scopehal-apps/compare/e2617799fdf2...fd4fa0d0de3e
<_whitenotifier-9> [scopehal-apps] azonenberg fd4fa0d - Fixed race condition when WaveformGroup could access areas in rendering thread that had already been removed in UI thread
<_whitenotifier-9> [scopehal-apps] bvernoux edited issue #589: ngscopeclient Windows crash when loading with "File => RecentFiles" (work fine when using "File => Open Offline...") - https://github.com/glscopeclient/scopehal-apps/issues/589
bvernoux has joined #scopehal
<azonenberg> bvernoux: tl;dr WaveformGroup could access waveform areas that had already been deleted
<azonenberg> because of lack of mutexing around the array of areas
<bvernoux> great fix
<bvernoux> I'm building it
<azonenberg> now i have locks in a few places where i really need them
<azonenberg> but otherwise i do mutexed atomic snapshots of the array and then unlock
<bvernoux> I have finished the port to clean C of mingw-bundledlls
<azonenberg> the idea being it's all shared pointers anywy
<azonenberg> it's OK if i render a waveform one frame after i delete it
<azonenberg> as long as all of the attached pointers remain valid for that frame
<azonenberg> wasted effort but not going to break anything
<azonenberg> After we deprecate glscopeclient i plan to refactor a bunch more stuff to use shared/unique pointers, like filter creation
<azonenberg> and maybe instrument creation
<azonenberg> that will make a lot of stuff simpler
<bvernoux> it is strange but it seems the Open Recent is also fixed
<azonenberg> May have been the same race condition
<azonenberg> Close them all i guess, can reopen the tickets if they come back?
<bvernoux> the slow down is not fixed on 2 steps
<bvernoux> exactly the same steps
<azonenberg> slowdown is not fixed? ok i'll troubleshoot further
<azonenberg> i had not intended to debug it yet :p
<azonenberg> i may have just got lucky with the one test i ran
<bvernoux> just checked now on WIn11
<bvernoux> Rasterize time >250ms
<bvernoux> Tone map time seems ok now about 140us
<azonenberg> I think what's happening is the same issue
<azonenberg> something holds a lock too long or that it doesnt need
<bvernoux> it is hard to say in fact as we do not have min/max for exch perf counter
<azonenberg> rasterize and tone map both separately acquire that lock
<azonenberg> so either one might hit it
<azonenberg> and yes min/max might not be bad to add
<bvernoux> So on Win11 Pro it is not fixed for me
<bvernoux> I need to switch to Ubuntu to check
<bvernoux> The Open Offline from recent is not fixed in fact ;)
<bvernoux> it seems also linked to a race condition
<_whitenotifier-9> [scopehal-apps] bvernoux commented on issue #590: ngscopeclient Windows / Linux(Ubuntu 23.04) big slow down on some zoom step - https://github.com/glscopeclient/scopehal-apps/issues/590#issuecomment-1565956465
<azonenberg> the joys of heavily threaded apps :p
<azonenberg> ok i'll look into it in the morning
<_whitenotifier-9> [scopehal-apps] bvernoux edited a comment on issue #590: ngscopeclient Windows / Linux(Ubuntu 23.04) big slow down on some zoom step - https://github.com/glscopeclient/scopehal-apps/issues/590#issuecomment-1565956465
<bvernoux> The hard part is Vulkan Validation / GDB does not help to track down such type of issue
<azonenberg> yeah and hangs are harder to fix than crashes
<azonenberg> since they self resolve
<azonenberg> i have an idea for a debug tool though
<azonenberg> basically, create a locker with a built in "should never take more than X time" check instead of std::lock_guard
<azonenberg> that does trylock with timeout in a loop
<azonenberg> and if X time has elaspsed and it still doesnt have the lock, triggers a breakpoint
<azonenberg> this should let me investigate what else might be holding it
<azonenberg> also vtune and nsight have some good tools for investigating lock behavior
<azonenberg> at least in terms of finding what you're spending a lot of time waiting on
<bvernoux> ha yes very good idea
<bvernoux> I have nsight installed in WSL2 (as it does not work on Windows IIRC)
<azonenberg> i've only ever used it native linux
<azonenberg> anyway, i'll poke at it. one segfault fixed is still progress
<bvernoux> Do you reproduce the slow down on your side ?
<bvernoux> As I can reproduce that with my new PC RTX4070 and also on my old PC laptop GeForce GT 650M
<bvernoux> Before I never reported that issue on the old PC because I was thinking it was NVidia Driver issue on that old HW
<bvernoux> and also before it was not possible to load scopession and provide full setup/test to reproduce it
<bvernoux> could you share other dataset with scopesession ? (including eye diagram, PCIe ...)
<bvernoux> I could do deeper test on different decoder like that too
<azonenberg> I dont think the hang is decode related. decodes only run when you load new data from the instrument or navigate history
<azonenberg> zooming doesn't rerun them
<azonenberg> hanging at certain zooms is clearly a rendering side issue
<azonenberg> its likely a thread sync issue between WaveformThread and the gui thread
<bvernoux> yes i just ask other dataset for other test
<bvernoux> I have tested the spi-compressed_data which works perfectly
<bvernoux> I'm rebuilding latest ngscopeclient on my old PC
<bvernoux> it just crashed with make -j ;)
<bvernoux> not enough memory
<bvernoux> make -j4 is mandatory with 16GB RAM
<bvernoux> for information on the Ryzen9 7950X3D with make -j memory consumption during build climb to >32GB
<bvernoux> with ninja the build is a bit faster too
<bvernoux> I'm pretty sure the build time could be improved too as there is lot of time when there is only 1 or 2 cores used
<bvernoux> on 2 minutes build all CPU cores are used during about 1 minute
<bvernoux> but it is not really an issue ;)
<azonenberg> So on large builds we bottleneck on linking
<azonenberg> i think there are things we can do to make that run faster
<azonenberg> in particular, for release builds turning debug info off
<azonenberg> the binaries are enormous right now and most of it is symbols
<bvernoux> Yes it is mainly link which is slow with huge binaries because of all symbols
<bvernoux> For information when ngscopeclient & glscopeclient are stripped with all dependencies full archive is < 100MB
<bvernoux> At least on Windows
<bvernoux> I'm not sure it is that easy to do a full archive without dependencies on Linux Ubuntu ...
<bvernoux> hmm on my old PC the effect is diffenrent
<bvernoux> the slow down appears on zoom out at start
<bvernoux> so it look like a limitation of the GFX card but the dataset is quite not complex so maybe it trig the bug too
<azonenberg> slowdown when zooming out that occurs with a high rasterize and low tone map time
<azonenberg> and does not reduce FPS except when panning or zooming?
<azonenberg> that's just your GPU not keeping up with the number of samples on screen
<azonenberg> i want to add some more optimizations for older GPUs to reduce level of detail and improve ui responsiveness in those situations
<azonenberg> but that's not a bug and definitely not the same issue as the hangs
<bvernoux> Yes for the old computer I suspect heavily it is limitation of the GPU ...
<azonenberg> yep we have an open ticket for some sort of LOD to improve performance with old GPUs
<azonenberg> havent had time to touch it. but that's not a bug, just too much data for the hardware
<azonenberg> Not same root cause at all
<bvernoux> Yes I confirm
<bvernoux> For those interested I have released https://github.com/bvernoux/mingw-bundledlls
<bvernoux> Rewritten from scratch from the original Python3 script https://github.com/bvernoux/mingw-bundledlls
<bvernoux> It is what I'm using internally for ngscopeclient.exe & glscopeclient.exe dependencies
<_whitenotifier-9> [scopehal-apps] bvernoux edited a comment on issue #569: Windows CI builds don't install properly due to missing GTK libs - https://github.com/glscopeclient/scopehal-apps/issues/569#issuecomment-1565253316
<_whitenotifier-9> [scopehal-apps] bvernoux edited a comment on issue #569: Windows CI builds don't install properly due to missing GTK libs - https://github.com/glscopeclient/scopehal-apps/issues/569#issuecomment-1565253316
<_whitenotifier-9> [scopehal-apps] bvernoux edited a comment on issue #569: Windows CI builds don't install properly due to missing GTK libs - https://github.com/glscopeclient/scopehal-apps/issues/569#issuecomment-1565253316
bvernoux has quit [Quit: Leaving]
josuah has quit [Quit: josuah]
josuah has joined #scopehal
juri__ has joined #scopehal
juri_ has quit [Ping timeout: 256 seconds]
juri__ has quit [Read error: Connection reset by peer]
<d1b2> <Aleksorsist> I'm about to do some testing on a Windows machine (my signal hound doesn't support Linux), can I just grab the windows portable from https://github.com/glscopeclient/scopehal-apps/actions/runs/5102972543 ? Or do I need to do something else to get it to work? I've looked at issue 569 but I don't quite understand it
juri_ has joined #scopehal
juri_ has quit [Read error: Connection reset by peer]
juri_ has joined #scopehal
bvernoux has joined #scopehal
<azonenberg> bvernoux: (09:42:32) d1b2: <Aleksorsist> I'm about to do some testing on a Windows machine (my signal hound doesn't support Linux), can I just grab the windows portable from https://github.com/glscopeclient/scopehal-apps/actions/runs/5102972543 ? Or do I need to do something else to get it to work? I've looked at issue 569 but I don't quite understand it
<azonenberg> i'm not testing on windows so cant comment on that
<d1b2> <Aleksorsist> Ok I just tried the portable. The following DLLs aren't found: libgmodule-2.0.0.dll, libfontconfig-1.dll,libfreetype-6.dll,libpixman-1-0
<d1b2> <Aleksorsist> This is a fresh install of windows btw, I will try installing MSYS2 now and see if that's got the DLLs I need
<azonenberg> Ot
<azonenberg> It's going to get a lot easier soon
<azonenberg> when we deprecate glscopeclient and stop using mingw
<azonenberg> the windows build will become far simpler
<d1b2> <Aleksorsist> Looking forward to that, I can do linux fine but the extra layer of MSYS2 for windows is throwing me for a loop
<d1b2> <Aleksorsist> I did install MSYS2 and no change for portable
bvernoux has quit [Remote host closed the connection]
<d1b2> <Aleksorsist> I'll try the .msi and if that doesn't work then I'll try to figure out building from source
<d1b2> <Aleksorsist> Ok, I still get a slew of DLLs missing
<azonenberg> yeah the from source build should work but is a bit of a pain to set up
bvernoux has joined #scopehal
<d1b2> <Aleksorsist> Been following the build from source guide and got this error while compiling glslang: In file included from MachineIndependent/glslang.y:86: C:\msys64\home\EEVengers-Lab0\glslang\glslang\MachineIndependent\SymbolTable.h:648:18: error: 'uint32_t' does not name a type
<azonenberg> hmmm, interesting
<azonenberg> that's a third party library, it should compile :p
<d1b2> <Aleksorsist> I added #include <cstdint> to line 71 of SymbolTable.h as the compiler suggested and it worked
<azonenberg> yeah that sounds like an upstream issue
bvernoux has quit [Read error: Connection reset by peer]
<d1b2> <Aleksorsist> I got it running, now to see if I can get waveforms in from ThunderScope
Degi has joined #scopehal
<whitequark> sometimes programs unintentionally rely on C headers implicitly included by other C headers
<whitequark> standard C headers, I mean
<whitequark> so when you change the C library (also happens with C++), like when you swap to Windows or macOS, you end up with this kind of issue
<Bird|ghosted> yeah, inadvertently depending on an indirect dep is kinda bad :P
<azonenberg> yeah we've fixed severla such issues in scopehal but i expected glslang to be better lol
<azonenberg> otoh, khronos also shipped a completely broken (didnt even compile) vulkan-hpp in one sdk release sooooo