azonenberg changed the topic of #scopehal to: libscopehal, libscopeprotocols, and glscopeclient development and testing | https://github.com/glscopeclient/scopehal-apps | Logs: https://libera.irclog.whitequark.org/scopehal
Degi_ has joined #scopehal
Degi has quit [Ping timeout: 265 seconds]
Degi_ is now known as Degi
<_whitenotifier-7> [scopehal] azonenberg pushed 1 commit to master [+0/-0/±1] https://github.com/glscopeclient/scopehal/compare/b4cf3783ddae...62825f3b3581
<_whitenotifier-7> [scopehal] azonenberg 62825f3 - Disabled GPU acceleration in TRCImportFilter since it seems to be broken. Will investigate after queue refactoring is done since a proper fix depends on a queue allocator.
<azonenberg> numerical precision issues in waveform rendering are back. unsure if refactoring bug or something else
<azonenberg> (in glscopeclient)
<azonenberg> investigating...
<azonenberg> Seems like it was introduced with the vulkan refactoring. that's a massive enough change it will be hard to bisect etc. Time to hav esome fun lol
<azonenberg> lain: so i figured out the plotRight issue while looking at this
<azonenberg> turns out you were doing the right thing
<lain> :O
<azonenberg> the texture is drawn over the entire gl viewport with no transformation
<azonenberg> essentially what we end up doing is, allocate the buffer the size of the whole window
<azonenberg> but only actually invoke the shader on the first plotRight columns
<azonenberg> so we just waste a little memory
<lain> ohh
<azonenberg> We could fix it, but ngscopeclient eliminates the issue entirely so not worth it
<azonenberg> right now i'm investigating what looks ilke weird numerical precision issues the GL shader doesnt have though
<lain> oo
<lain> innteresting
<azonenberg> this is a significant regression so probably worth you spending time troubleshooting if you can stash your WIP mac stuff?
<azonenberg> i dumped a test case in ark:/tmp/
<azonenberg> foo.scopesession plus the two .trc files
<lain> kk
<lain> I'll look into that tomorrow
<azonenberg> it should open to a view that shows off the failure mode
<azonenberg> ok
<azonenberg> (This is bad enough i'm not going to be demoing it like this on my trip. Worst case i can check out a pre-vulkan commit for that purpose)
<azonenberg> i just noticed it while going through my library of scopesession's looking for cool things to show
<lain> yeah this'll be fun to bisect lol
<azonenberg> (can you grab the test case quick and confirm it opens / exhibits the failure condition for you?)
<azonenberg> i patched the scopesession to use a relative path for the import files so you'll need to have pwd be whatever you dumped the files into
<lain> ah, 1sec
<azonenberg> aaand now qsgmii is segfaulting for unrelated reasons, yaaay
<lain> we need more tests :P
<azonenberg> we do :p
<azonenberg> i have a large corpus of test data but no automated testing attached to it
<lain> well
<lain> in glscopeclient on macos on my m1 it hits an unhandled exception
<lain> so that's fun
<azonenberg> what's the exception?
<azonenberg> it's probably failing to find some file because you have it in the wrong directory?
<azonenberg> like i said i patched the test case to be a relative path
<lain> nope
<lain> it's vulkan-related
<azonenberg> :o
<lain> also the foo.scopesession was using an absolute path on your /ceph/ but I edited it to use an absolute path to where I dumped the files locally
<azonenberg> ah i swear i patched it
<azonenberg> must have been after i uploaded it lol
<lain> oh, fascinating
<lain> if I fire up vkconfig, it didn't crash this time
<lain> plenty of messages to help me narrow down what broke
<lain> azonenberg: actually my first time loading a scopesession file, should the waveforms show up immediately or do I have to hit a button?
<azonenberg> it should show up immediately
<azonenberg> oh wait
<azonenberg> you might have hit a bug i just patched
<azonenberg> grab 62825f3 in scopehal
<azonenberg> (latest)
<lain> lol
<lain> does that include my macos patches?
<azonenberg> that's scopehal side
<lain> or I guess you mean merge those huh
<azonenberg> you can cherrypick that commit
<azonenberg> it shoudlnt conflict
<azonenberg> so it seems an AcceleratorBuffer containing a std::vector is a bad idea
<azonenberg> i need to troubleshoot more
<lain> oh?
<azonenberg> AcceleratorBuffer generally expects to work with POD datatypes
<azonenberg> i'm not sure exactly how it's going sideways yet
<azonenberg> but it probably won't handle anything contianing pointers to itself well
<azonenberg> for starters
<lain> ok building glscopeclient with the changes
<lain> I just did a git merge origin/master because I was already close enough that it was seamless
<lain> HEY
<lain> I have waveofrms
<lain> waveforms*
<lain> unrelated, the sleepy medicine is kicking in so this is gonna go sideways real fast
<lain> but yes this appears to exhibit the bug, screenshot inbound to confirm
<_whitenotifier-7> [scopehal] azonenberg labeled issue #704: Regression: all Ethernet protocol decodes crash - https://github.com/glscopeclient/scopehal/issues/704
<_whitenotifier-7> [scopehal] azonenberg opened issue #704: Regression: all Ethernet protocol decodes crash - https://github.com/glscopeclient/scopehal/issues/704
<lain> pls confirm but I assume C2-C1 shouldn't look like that
<lain> that would be SubtractFilter, yeah?
<lain> anyway I'll bisect this on my x86_64 box tomorrow so I don't have to deal with breaking macos compat during the bisect
<azonenberg> so that is actually a TIE filter
<azonenberg> looking at clock jitter of C2-C1 wrt the PLL running on them
<lain> ohh
<azonenberg> it's a sparse waveform
<azonenberg> with femtosecond timescale
<lain> wonderful
<azonenberg> which means that all of the timestamp values are enormous
<azonenberg> this used to work, now it doesn't
<azonenberg> if i had to guess, something is being cast to fp32 that used to be done in the int64 domain
<lain> probably a bug in the shader or invoking it is my guess, but yeah, bisect will tell all
<azonenberg> we need to be extremely careful with type promotions when working with timestamps
<lain> aye
<azonenberg> keep as much as possible in the int64 domain and convert to float for rendering at the last possible moment
<lain> it's also plausible something behaves differently in vulkan
<lain> shaders
<azonenberg> That is also plausible
<lain> anyway, I'll get the nvidia debug stuff installed on my amd box and get this sorted :3
<azonenberg> anyway, i dont care the bug is in your changes or vulkan's handling of math or what
<azonenberg> but yeah
<lain> but for now, the sleepy meds have kicked in so I'm lucky to be typing words :P
<lain> nini
massi has joined #scopehal
tiltmesenpai has quit [Read error: Connection reset by peer]
tiltmesenpai has joined #scopehal
<_whitenotifier-7> [scopehal-apps] RX14 commented on issue #507: Link error with reflowmon - https://github.com/glscopeclient/scopehal-apps/issues/507#issuecomment-1266784220
massi has quit [Remote host closed the connection]
massi has joined #scopehal
bvernoux has joined #scopehal
<_whitenotifier-7> [scopehal] azonenberg pushed 1 commit to master [+0/-0/±2] https://github.com/glscopeclient/scopehal/compare/62825f3b3581...42791c660448
<_whitenotifier-7> [scopehal] azonenberg 42791c6 - AcceleratorBuffer: allow non trivially copyable types, but only in CPU-only buffers
* lain wakes up
<_whitenotifier-7> [scopehal] azonenberg pushed 1 commit to master [+0/-0/±1] https://github.com/glscopeclient/scopehal/compare/42791c660448...3aeb074fe095
<_whitenotifier-7> [scopehal] azonenberg 3aeb074 - VectorFrequencyFilter: fixed missing resize call
<_whitenotifier-7> [scopehal] azonenberg closed issue #704: Regression: all Ethernet protocol decodes crash - https://github.com/glscopeclient/scopehal/issues/704
<_whitenotifier-7> [scopehal] azonenberg commented on issue #704: Regression: all Ethernet protocol decodes crash - https://github.com/glscopeclient/scopehal/issues/704#issuecomment-1267221287
<lain> hm ok
<lain> well
<lain> my x86_64 box isn't rendering *anything* when I open that scopesession from master, so that's neat :D
<azonenberg> lol lovely
<azonenberg> let me check what happens for me
<_whitenotifier-7> [scopehal-apps] azonenberg pushed 4 commits to master [+2/-0/±32] https://github.com/glscopeclient/scopehal-apps/compare/1212cb5bced6...d8627a0b1443
<_whitenotifier-7> [scopehal-apps] azonenberg 6199908 - Initial implementation of preferences dialog in ngscopeclient. Supports boolean and enum preferences only. See #522.
<_whitenotifier-7> [scopehal-apps] azonenberg e085a48 - Lots of work on preferences system in ngscopeclient
<_whitenotifier-7> [scopehal-apps] azonenberg 14e2e26 - Removed comment about unnecessary fix
<_whitenotifier-7> [scopehal-apps] azonenberg d8627a0 - Updated submodules. Fixed alpha calculation in ngscopeclient.
<azonenberg> lain: try with d8627a0 (latest master)
<azonenberg> it renders for me
<azonenberg> i fixed a few things in AcceleratorBuffer for non-POD datatypes recently
<azonenberg> (also i fixed a bug in the trc import filter recently that you might not have merged yet)
mikolajw has quit [*.net *.split]
whitequark has quit [*.net *.split]
asy_ has quit [*.net *.split]
jevinskie[m] has quit [*.net *.split]
lain has quit [*.net *.split]
sajattack[m] has quit [*.net *.split]
miek has quit [*.net *.split]
JSharp has quit [*.net *.split]
lethalbit has quit [*.net *.split]
t4nk_fn has quit [*.net *.split]
gruetzkopf has quit [*.net *.split]
ericonr has quit [*.net *.split]
sorear has quit [*.net *.split]
agg has quit [*.net *.split]
josuah has quit [*.net *.split]
electronic_eel has quit [*.net *.split]
bvernoux has quit [*.net *.split]
benishor has quit [*.net *.split]
massi has quit [*.net *.split]
tiltmesenpai has quit [*.net *.split]
bgamari has quit [*.net *.split]
Yamakaja has quit [*.net *.split]
monochroma has quit [*.net *.split]
Fridtjof has quit [*.net *.split]
esden has quit [*.net *.split]
tnt has quit [*.net *.split]
anuejn has quit [*.net *.split]
vup has quit [*.net *.split]
_florent_ has quit [*.net *.split]
welterde has quit [*.net *.split]
elms has quit [*.net *.split]
kbeckmann has quit [*.net *.split]
florolf has quit [*.net *.split]
mxshift has quit [*.net *.split]
Stary has quit [*.net *.split]
balrog has quit [*.net *.split]
d1b2 has quit [*.net *.split]
Stephie has quit [*.net *.split]
kbeckmann has joined #scopehal
esden has joined #scopehal
Yamakaja has joined #scopehal
bvernoux has joined #scopehal
massi has joined #scopehal
tiltmesenpai has joined #scopehal
lain has joined #scopehal
balrog has joined #scopehal
jevinskie[m] has joined #scopehal
whitequark has joined #scopehal
mikolajw has joined #scopehal
sajattack[m] has joined #scopehal
agg has joined #scopehal
asy_ has joined #scopehal
josuah has joined #scopehal
electronic_eel has joined #scopehal
t4nk_fn has joined #scopehal
bgamari has joined #scopehal
lethalbit has joined #scopehal
miek has joined #scopehal
JSharp has joined #scopehal
_florent_ has joined #scopehal
vup has joined #scopehal
tnt has joined #scopehal
welterde has joined #scopehal
florolf has joined #scopehal
elms has joined #scopehal
Stary has joined #scopehal
Stephie has joined #scopehal
d1b2 has joined #scopehal
anuejn has joined #scopehal
sorear has joined #scopehal
ericonr has joined #scopehal
gruetzkopf has joined #scopehal
monochroma has joined #scopehal
benishor has joined #scopehal
Fridtjof has joined #scopehal
mxshift has joined #scopehal
<lain> azonenberg: wb
<azonenberg> yay netsplits
<azonenberg> anyway did the latest master from a few minutes ago render for you?
<lain> lemme test
<lain> buildy buildy
<lain> buildy buildy :3
<lain> yep fixed
<lain> what'd you change? or should I just read the commit logs :P
<azonenberg> among other things i temporarily disabled the vulkan implementation of the int8/int16 -> fp32 conversion filter
<azonenberg> which is used by the .trc importer
<azonenberg> i dont know why it stopped working, it used to work and i didnt change anything
<azonenberg> but it didn't actually save us that much run time
<lain> oh I thought you already did that last night
<azonenberg> I did
<azonenberg> but i hadn't pushed the new submodule pointer to scopehal-apps
<lain> ahhhh ok
<azonenberg> because i had incomplete stuff in my working copy and didnt want to have to cherrypick part of it to push
<lain> gotcha
<_whitenotifier-7> [scopehal-apps] azonenberg opened issue #526: Figure out how to handle multiple defaults for colors depending on GUI theme - https://github.com/glscopeclient/scopehal-apps/issues/526
<_whitenotifier-7> [scopehal-apps] azonenberg labeled issue #526: Figure out how to handle multiple defaults for colors depending on GUI theme - https://github.com/glscopeclient/scopehal-apps/issues/526
massi has quit [Remote host closed the connection]
<lain> azonenberg: so I'm actually having trouble finding a previous commit that works as intended
<azonenberg> lain: try a10fe6c
<lain> doesn't work
<lain> I'll rebuild to see why it didn't work, but that was the first one I tried
<azonenberg> huh
<azonenberg> i swear i tried that last night and it was ok
<lain> will know in a few minutes!
<azonenberg> anyway, i just want a fix. i'd suggest focusing more on root cause than diagnosing a potential regression
<azonenberg> treat it as a new bug
<lain> hm ok
<lain> I was hoping to use bisect to narrow down the introduction point
<azonenberg> yeah i know
<lain> Warning: ReadDataFile: Could not open file "shaders/Convert16BitSamples.spv"
<lain> terminate called after throwing an instance of 'vk::UnknownError'
<lain> what(): vkCreateShaderModule: ErrorUnknown
<lain> fish: “./src/glscopeclient/glscopeclie…” terminated by signal SIGABRT (Abort)
<azonenberg> lol, ok
<azonenberg> so that's the import filter not being happy because of missing shader config
<azonenberg> you can probably just comment out the gpu code in TRCImportFilter as a quick workaround?
<lain> mmm it just segfaults
<azonenberg> or also try updating scopehal to latest
<azonenberg> and only using the old scopehal-apps code
<lain> oh, I patched it badly
<azonenberg> ah ok
<azonenberg> that would do it
<lain> eh, ok, this has bugs from the vulkan render shaders in it
<lain> I'mmm just gonna examine this as a new bug as you suggested :P
<azonenberg> yeah
<azonenberg> i feel like we've made extensive enough changes trying to bisect would be a nightmare and the diff would essentially be the entire shader rewrite
<lain> yeahhh
<azonenberg> as a starting point, i would be very suspicious of all math on x axis values especially if there is any chance of being cast to float32 at some point in the process
<azonenberg> i attempted to keep stuff in the int64 domain as long as possible such that after the offset, the dynamic range from left to right of a single plot would easily fit in fp32
<azonenberg> but that may have gone wrong somewhere
<azonenberg> as a first order experiment, i suggest patching some code at various points to act as if the samples were uniform
<azonenberg> essentially invent a fictional timebase and ignore the actual offset values in the code
<azonenberg> see what happens
<azonenberg> but i'm not sure if that will tell you anything useful
* lain nod
<azonenberg> it will at least let you confirm the problem is wrt x axis math i guess
<lain> true, yeah
<lain> azonenberg: just to confirm, this is how it SHOULD look, right? https://cdn.discordapp.com/attachments/776941750291267595/1026712786895642714/precision.png
<lain> or is that also exhibiting the bug
<azonenberg> That giant gap at left should not be there
<lain> ah ok
<azonenberg> you'll notice as you zoom in it comes and goes
<azonenberg> and some samples appear and disapppear
<azonenberg> i think due to it miscalculating the left/right bounds of which samples go in which pixels
<lain> yep
<azonenberg> so some columns of pixels get no samples or something
<azonenberg> It would not surprise me if at least some of the bug is in PrepareGeometry() rather than the actual shader
<azonenberg> We had a similar bug some time ago and iirc converting "xscale" from fp32 to fp64 fixed it
<azonenberg> but it's fp64 now and we still have the issue
<azonenberg> but that may not have been a complete fix, idk
<lain> hmm
<lain> so I added some quick debug code to WaveformArea::RenderTrace, to just LogDebug whether data->IsDensePacked() is true or false
<lain> should there be multiple waveforms in the C2-C1 WaveformArea?
<lain> oh nvm I misread the output
<azonenberg> The top plot is the uniform analog differential input signal (computed by subtracting two signals from the trc import filter) with three digital overlays
<azonenberg> there's a threshold which is uniform as well i think
<azonenberg> the CDR PLL which is sparse
<azonenberg> and the PRBS check which is sparse and should be all zeroes (no PRBS errors)
octorian has quit [Ping timeout: 244 seconds]
<azonenberg> the second plot is the extracted cycle to cycle jitter and should be one sample per UI, sparse analog
octorian_ has joined #scopehal
octorian_ is now known as octorian
<lain> hmm
<lain> I'm suspicious of the zero hold stuff
<lain> still debugging
<azonenberg> yes, i was too
<azonenberg> it seemed to work ok when i tested prior to merging
<azonenberg> but i did not, in retrospect, test with any extremely long 1fs resolution sparse waveforms
<azonenberg> Which is where numerical precision errors would creep in
<lain> ah, I found a typo that would cause unpredictable behavior
<azonenberg> ooh
<_whitenotifier-7> [scopehal-apps] azonenberg pushed 1 commit to master [+0/-0/±4] https://github.com/glscopeclient/scopehal-apps/compare/d8627a0b1443...bf3c3aaf731c
<_whitenotifier-7> [scopehal-apps] azonenberg bf3c3aa - Added support for preferences of "color" type in ngscopeclient. Added prefs for grid configuration. Improved display of grid to avoid colliding labels. See #522.
massi has joined #scopehal
<azonenberg> from your refactoring or the zhold patches?
<lain> not *the* bug, but a bug
<lain> zhold patches
<lain> looks like this should be sdigdat->m_durations
<azonenberg> yes, it does indeed look like it
<lain> I guess that wouldn't affect M1 though because unified memory :P
<azonenberg> no it does
<azonenberg> we are not using it as unified memory
<azonenberg> we have two different buffers we copy between
<azonenberg> they happen to be in the same address space
<azonenberg> but it's still two incoherent blocks of memory
<azonenberg> that need explicit synicng
<azonenberg> once we optimize to use a single buffer that's a different story
<_whitenotifier-7> [scopehal-apps] azonenberg pushed 1 commit to master [+0/-0/±1] https://github.com/glscopeclient/scopehal-apps/compare/bf3c3aaf731c...4250efe5e717
<_whitenotifier-7> [scopehal-apps] azonenberg 4250efe - Fixed copy-paste bug with duration values
<azonenberg> anyway just pushed a fix for that
<azonenberg> so i just realized something that further points to rounding or numerical precision issues
<azonenberg> Open the test case, click on the timeline, and drag it left/right veeery slowly
<azonenberg> note what looks like aliasing behavior
<azonenberg> pixels come and go as the waveform moves
<azonenberg> and the areas we render seem to shift as the x offset shifts
<azonenberg> you can also see that the waveforms have an area at the left that always renders correctly, then things go haywire
<azonenberg> aaand guess what
<lain> yep
<lain> :o
<lain> did you find it lol
<azonenberg> no
<azonenberg> but i have more evidence
<azonenberg> the first problems seem to crop up just about 2.5 ns
<azonenberg> hmm, nvm. i was off by a few OOMs
<azonenberg> i was thinking 2^31 fs
<azonenberg> but that's 2147 ns
<azonenberg> But, 2^23 fs is only 8 ns
<azonenberg> (23 bit mantissa in ieee754 fp32)
<azonenberg> which is the right order of magntiude for where we see problems
<azonenberg> if we do a bunch of multiplies and divides i could totally see rounding errors becoming >1 pixel at that scale
<lain> hmm
<lain> azonenberg: likely unrelated, but I'm looking at the localSize calculation at the start of WaveformArea::RenderTrace, it says it must match COLS_PER_BLOCK in waveform-compute shader, but there is no COLS_PER_BLOCK in there, so I'm guessing this code is obsolete
<lain> looks like it's just calculating numGroups, which becomes the number of x invocations for the shader
<azonenberg> let me look, h/o
<azonenberg> that is for the *y* axis size iirc
<azonenberg> or wait
<azonenberg> oh ok. yeah
<azonenberg> it's mostly obsolete. we used to support two or more columns of pixels in one shader thread
<azonenberg> one shader group*
<azonenberg> also, here's an interesting finding
<lain> ah ok
<azonenberg> I set indexBuffer to {0}
<azonenberg> i.e. every thread starts rendering from x=0
<azonenberg> this is of course stupidly inefficient as we spend a ton of time transforming samples we'll never see
<azonenberg> but more interesting was the outcome
<lain> er, hm
<azonenberg> everything renders perfectly, then it stops right where it would have gone bad
<lain> isn't indexBuffer set in PrepareGeometry?
<azonenberg> Yes
<lain> oh I see you're syaing you just set it to {0} as a test
<azonenberg> i patched out the BinarySearchForGequal call to return constant zero
<lain> fascinating o.o
* lain thinks
<azonenberg> i would have expected it to draw the entire waveform, just very slowly
<lain> indeed
<azonenberg> fwiw, there is apparently the ability to printf or log messages from shader code in vulkan using the validation layers
<lain> that smells like a shader issue then
<azonenberg> i havent used it
<azonenberg> but it may be worth trying
<azonenberg> it seems that this is mutually exclusive with gpu assisted bounds checking etc
<azonenberg> but we dont seem to be going OOB here
<azonenberg> as none of those checks are firing
<lain> azonenberg: does your gpu have int64 support? wondering if I can ignore !HAS_INT64 issues
<lain> iirc my M1 GPU does not have int64, so it's using the more annoying path
<azonenberg> yes
<lain> kk
<azonenberg> your nvidia should too i think
<lain> seems likely
<lain> I'm testing on the M1 currently since it doesn't seem to make a difference which machine I test on, and the nvidia machine is back in WA and VNC is slow :P
<_whitenotifier-7> [scopehal] azonenberg pushed 1 commit to master [+0/-0/±1] https://github.com/glscopeclient/scopehal/compare/3aeb074fe095...b10a70083a4d
<_whitenotifier-7> [scopehal] azonenberg b10a700 - Request VK_KHR_shader_non_semantic_info if available (required for shader based debug printf)
<azonenberg> ok so to get printf output from shaders, enable it in the vulkan config tool
<azonenberg> add #extension GL_EXT_debug_printf : require to the shader
<azonenberg> then call debugPrintfEXT() from the shader
<azonenberg> i need to get back to $dayjob stuff so bbl
<lain> kk
<azonenberg> and also pull latest (b10a700) scopehal
<azonenberg> which requests a device extension the printf requires
<lain> yeah, already did.. it's complaining of: program_source:88:9: error: use of undeclared identifier 'debugPrintfEXT'
<lain> currently debugging
<azonenberg> did you add that extension?
<lain> yep
<azonenberg> huh because i just did that on my end and it worked fine
<azonenberg> is your glslc too old maybe?
<lain> well this is happening at runtime
<lain> it compiles fine at build
<azonenberg> um
<azonenberg> at run time??
<azonenberg> i wonder if moltenvk doesnt implement that extension
<lain> hrm
<azonenberg> see if it says anything about VK_KHR_shader_non_semantic_info in the log
<azonenberg> or in vulkaninfo
<azonenberg> (device extension not instance extension)
<azonenberg> You might have to try this on the nvidia card. the nvidia should have it
<lain> hmm yeah I don't see it in there
<lain> alright
<azonenberg> yeah so its a moltenvk limitation then
<_whitenotifier-7> [scopehal] azonenberg commented on issue #688: "Unrecognized dataset type" when importing Tek .WFM file - https://github.com/glscopeclient/scopehal/issues/688#issuecomment-1267441628
<_whitenotifier-7> [scopehal] azonenberg closed issue #688: "Unrecognized dataset type" when importing Tek .WFM file - https://github.com/glscopeclient/scopehal/issues/688
<d1b2> <Mughees> Is Virtual machine supported for glscope?
<azonenberg> mughees: yes and no
<azonenberg> if you just throw together a VM, it probably won't work well/fast
<azonenberg> GPU acceleration in VMs tends to not be great
<azonenberg> with pcie passthrough of a dedicated GPU, it should work fine
<azonenberg> software emulation using swiftshader may be an option but we have not tested
<d1b2> <Mughees> alright...actually my hard disk gone bad..so had to switch to a windows laptop
<azonenberg> software emulation using llvmpipe will work out of the box eventually (although slow) but last time we tried we had some issues with it
<d1b2> <david.rysk> virgl may work
<d1b2> <david.rysk> If your host supports it
<d1b2> <Mughees> it does kind a work..but crashes while genrating a sinewave
<azonenberg> glscopeclient does build/run on windows also, although the build is a bit of a mess right now
<d1b2> <david.rysk> IIRC it worked last time I tried, was just not so fast
<d1b2> <Mughees> In windows, are we able to edit code within MSYS2 environemnt?
<azonenberg> We're working on improving that. but the best path forward involves leaving GTK and moving entirely to ngscopeclient. which is several months from being at feature parity with glscopeclient
<azonenberg> msys2/mingw files are just normal files on your windows disk
<azonenberg> that you can edit with your text editor of choice
<d1b2> <Mughees> ok
<azonenberg> so you can edit the code there then compile in msys2
<d1b2> <Mughees> I think it is better to oreder a new hard 🙂
<azonenberg> We want to transition away from msys2 and just generate visual studio projects with cmake
<azonenberg> but while doing that with GTK is not impossible, it's a huge pain
<azonenberg> One of about a dozen reasons we're leaving GTK
<lain> ok there is definitely something weird in the shader
<lain> still debugging, but I'm seeing some nonsense values from the push constants
<azonenberg> struct packing issue?
<lain> quite possibly, hm
<lain> innerXoff=-1542624 window=4294967295x2073 memDepth=180 offset_samples=645832 alpha=0.000000 xoff=2.000000 xscale=0.000000 ybase=0.000113 yscale=90.000000 yoff=0.014734 persistScale=192.500000
<azonenberg> might also be 64 bit ints not handled by debugprintfext
<azonenberg> may need to explicitly print low/high halves or something?
<lain> ah true I should check the docs for the format specifiers
<lain> >No length modifiers. Everything except ul, lu, and lx is 32 bits, and ul and lx values are printed in hex
<azonenberg> is lu 64 bits?
<lain> it doesn't say, but that's my guess
<lain> but wouldn't lu be unsigned? whereas I have an int64_t
<lain> hm
<azonenberg> so there's no ld
<lain> seems not
<lain> well I can use %lx at least
<azonenberg> yeah
<lain> okay yeah it was just an issue with that
* lain back to hunting!
<lain> this may be a quirk of debugPrintfEXT, but I'm only ever seeing output from gl_GlobalInvocationID.x == 0, despite numGroups=1969
<lain> oh hm
<azonenberg> i got output from everything when i tried
<azonenberg> you do have to set the buffer size in the validation box
<azonenberg> it can be up to a megabyte
<lain> ohh there we go
<lain> thx
<bvernoux> it seems limit of GitHub are reached
<bvernoux> I suspect it is because there is not enough disk space ...
<azonenberg> Yes
<azonenberg> Yet another reason to move to our own local CI runners in the near future
<lain> hmm
<lain> oh, that's interesting
<lain> InterpolateY isn't using vec2 right at all
<azonenberg> ?
<lain> float InterpolateY(vec2 left, vec2 right, float slope, float x)
<lain> {
<lain> return left.y + ( (x - left.x) * slope );
<lain> }
<lain> so far I see no evidence of numerical instability, I'm beginning to wonder if this is a logic error
<azonenberg> very possible
<azonenberg> All i can say with certainty at this point is it's a bug :p
<azonenberg> try printing out the actual digital samples you process around x=500 or wherever the big gap is in the CDR trace
<azonenberg> look at start/end, see if you are taking an early-out too soon or being passed incorrect index data
<bvernoux> nice latest ngscopeclient rendering in demo is even faster
<bvernoux> I have about 58fps in full screen
<bvernoux> before it was slower with framerate between 48fps to 56gps
<bvernoux> nice there is the cursor too
<bvernoux> which work fine
<bvernoux> and the menu is instant even if so far there is only that option it is day and night vs glscopeclient right menu click
<azonenberg> i mean there isnt much there. glscopeclient does a lot more work to set up the menu
<azonenberg> but we will definitely be caching heavily
<lain> azonenberg: time ticks are, what, femtoseconds?
<azonenberg> in a normal time domain waveform, yes
<azonenberg> in frequency domain, Hz
<azonenberg> although we will likely shift to mHz or uHz to provide better resolution in the future
<lain> hrm
<lain> not seeing anything weird with the voltage values, interesting...
* lain tests some ground truths
<azonenberg> are you looking at the cdr pll or what?
<azonenberg> the pll output is digital so less things to go wrong i think
<azonenberg> probably easier to test there
<lain> hm ok
<lain> I've been looking at the TIE measurement
<lain> oooh ok
<lain> getting somewhere now...
<azonenberg> oh?
<azonenberg> are you seeing some pixels that dont render any samples?
<azonenberg> because i'm very sure that's what is happening
<azonenberg> i just dont know *why*
<lain> yes
<lain> I replaced voltage[i] with a fixed value in the calculations for left and right in the main loop
<lain> I would expect to see a straight line
<azonenberg> And you don't?
<lain> it is a straight line but it has the missing pixels
<azonenberg> try printf'ing the total number of samples drawn in each column of pixels
<lain> I think you're right that it's an issue in the x calculation somewhere
<azonenberg> i bet you'll see some threads that do nothing
<lain> ahm
<lain> azonenberg: wait how would you calculate that in the shader?
<lain> looks like g_updating[gl_LocalInvocationID.y] gets set true if that pixel was updated, else false
<azonenberg> Yeah
<lain> ok yeah
<azonenberg> so waht you'd do is add a bit of logic in thread y=0 of the block
<azonenberg> that sums g_updating each pass through the main loop
<azonenberg> into a local
<azonenberg> records a total count of how many pixels that thread wrote to
<azonenberg> then after the end of the loop, printf it
<azonenberg> (only in y=0)
<azonenberg> since you dont care which *threads* handled the samples
<azonenberg> you just care about the whole block drawing zilch
massi has quit [Remote host closed the connection]
<lain> ok I'm forgetting how this works, would I use the global or local invocation's y to check for == 0 ?
<lain> or I can just look it up in a sec
<lain> but brb, dr appt for a bit
<azonenberg> local
<azonenberg> local is thread index within the block
<lain> ok back
<lain> azonenberg: ok so I added a 'shared uint total_updated;', and then at the top of main(), if local y=0 I set total_updated = 0;...
<lain> near the end of the main loop there's an if(g_updating[y]) { ... }, and I increment total_updated in there, and then after the main loop, if this is y==0, I print the value
<lain> does that sounds right?
<lain> and indeed I'm getting plenty of zeroes in the output
<lain> ok, so why aren't some threads doing anything... hm hmmmmmm!
<azonenberg> yeah that sounds about right. you might have some race conditions if you dont use atomic shareds or something
<azonenberg> but for this purpose it should be good enough
<azonenberg> actually total_updated doesnt even have to be shared
<azonenberg> you can just only write it in y=0
<lain> oh, true
<azonenberg> just make sure you check if g_updating[] i true for any Y
<azonenberg> anyway, point is you're getting lots of threads doing nothing, just as i suspected
<azonenberg> the why, i havent got a clue :p
<lain> so I just added a check
<lain> hmm
<lain> narrowing down what path it's taking...
<lain> ok it's hitting this path a bunch
<lain> //Skip offscreen samples
<lain> if( (right.x >= gl_GlobalInvocationID.x) && (left.x <= gl_GlobalInvocationID.x + 1) )
<lain> when that is false, the thread doesn't draw
<lain> and it seems to be false very often
<lain> wait
<lain> is that reversed lol
<azonenberg> Sounds fishy
<azonenberg> let's think
<azonenberg> we want the sample to start before the right edge of our pixel
<azonenberg> and end after the left edge
<azonenberg> so that sounds correct?
<azonenberg> is this with you using the proper index, or hardcoded to zero?
<lain> proper index
<azonenberg> Patch the index to zero to eliminate it as a variable
<azonenberg> the index should only be a speed optimization
<azonenberg> if it changes behavior - which it does - that implies we have a different bug
<azonenberg> let's focus on one bug at a time
<lain> ok so essentially replace xind[...] with 0 ?
<lain> err no that can't be right?
<azonenberg> I'd do it in preparegeometry to avoid making too many changes to the shader
<lain> it's only used in two places
<azonenberg> indexBuffer[j] = 0
<azonenberg> the index is just the starting point for the thread
bvernoux has quit [Quit: Leaving]
<lain> uint iend = xind[gl_GlobalInvocationID.x + 1];
<lain> if(iend <= 0)
<lain> g_done = true;
<lain> hrm
<lain> wouldn't this set g_done=true right away though?
<azonenberg> yeah i'm thinking
<azonenberg> ok my mistake
<azonenberg> I'm not sure what that check is for
<azonenberg> oh
<azonenberg> this is to skip drawing samples before the 0th
<lain> shouldn't it be < rather than <= ?
<azonenberg> We currently fill index=0 from the left pixel in the plot up to the first sample that actually contains waveform data
<azonenberg> it would probably be better to fill with a negative value and change to -
<azonenberg> change to < 0
<azonenberg> see WaveformArea::BinarySearchForGequal()
<azonenberg> if we're searching for a value before the start of the waveform it returns 0
<azonenberg> (feel free to comment the renderer more aggressively, this is some of the heavily optimized arcane black magic that makes glscopeclient tick lol)
<lain> haha
<lain> hmm
<azonenberg> yep. giant gaps that shouldnt be there
<lain> this is such a weird bug lol
<lain> ok
<lain> so I'm going to patch voltage to 0 and indexes to 0
<azonenberg> and disable that early out for iend <= 0?
<lain> yeah
<lain> ok that's interesting
<azonenberg> ?
<lain> I haven't patched voltage=0 yet, but with index=0 and getting rid of that early out on iend <= 0, it seems to be working as expected
<lain> ok so there's something wrong with the logic there
<lain> or a larger problem lurking somewhere :P
<azonenberg> lol
<azonenberg> what happens if you *just* get rid of the early out
<azonenberg> and keep the real indexes
<lain> testing
<lain> I suspect this will work, I think the early out is the reason some threads were being lazy
<lain> yeah, this works fine
<lain> ok so it's that early out.
<azonenberg> dump the indexes?
<lain> sure
<azonenberg> maybe they're corrupted somehow
<azonenberg> They should be monotonic left to right with no 0s
<lain> I'll dump them from the shader and c++ to be extra sure
<azonenberg> all 0 from start to first sample, then increasing
<azonenberg> ok
<lain> OH
<lain> ohno.
<lain> the ... ok.
<lain> AcceleratorBuffer<int64_t> m_indexBuffer;
<lain> vs
<lain> uint xind[];
<azonenberg> um
<lain> that doesn't need to be an int64...
<azonenberg> that's no good lol
<azonenberg> So actually
<azonenberg> the fix should be on the host side
<azonenberg> using 32 bit indexes
<lain> yeah
<azonenberg> Because Vulkan has a 4GB memory allocation cap
<lain> ffs.
<azonenberg> so we can't have a waveform >1G point without major refactoring to split them into multiple buffers
<azonenberg> Sooo was this a refactoring regression?
<lain> I think so, I'm gonna go check when I introduced that and what it replaced to be sure, but I think so.
<azonenberg> anyway, good find
<azonenberg> awaiting PR once you confirm the fix :)
<lain> yep there it is
<lain> uint32_t*m_mappedIndexBuffer;
<lain> that was the original decl
<lain> and when moving it to AcceleratorBuffer I copypasta failed
<azonenberg> good luck finding that in a bisect lol
<lain> lol yeah jeez
<lain> welp.
* lain discards her debug changes, prepares a PR
<lain> actually it's a single commit, one-line change, mind if I just push to master?
<_whitenotifier-7> [scopehal-apps] azonenberg pushed 1 commit to master [+3/-0/±5] https://github.com/glscopeclient/scopehal-apps/compare/4250efe5e717...392e07afc132
<_whitenotifier-7> [scopehal-apps] azonenberg 392e07a - Initial work on font manager. Not yet integrated with preferences.
<azonenberg> Go for it
<_whitenotifier-7> [scopehal-apps] lainy pushed 1 commit to master [+0/-0/±1] https://github.com/glscopeclient/scopehal-apps/compare/392e07afc132...02b3c7f386b9
<_whitenotifier-7> [scopehal-apps] lain 02b3c7f - Fix regression for sparse waveform rendering.
<azonenberg> lain: fix confirmed
<azonenberg> thanks. back to macos stuff now? you said your branch is ready to check out and merge?
<lain> yeah I'm just merging master again right now as there's some changes I need to fix up
<lain> hrm
<lain> azonenberg: ok so TextureManager and Texture need access to a queue and cmdbuf under the new queue manager model. iirc the existing code just uses globals. is it safe to have them use the global transfer queue, or does it *need* to be the same render queue used by the main window?
<azonenberg> The global transfer queue needs to go away
<azonenberg> Because we need to support operating on intel iGPUs that have only one queue
<lain> alternatively I can have TextureManager grab a queue of its own, make a command pool, and a command buffer... but then locking access to the cmdbuf is mildly annoying
<azonenberg> ok let me rephrase
<azonenberg> we cannot dedicate a queue to transfer permanently
<azonenberg> so it will have to be a QueueHandle
<lain> yes
<azonenberg> Using the global transfer queue should be fine
<lain> so in my branch, g_vkTransferQueue is a QueueHandle
<azonenberg> Where it gets challenging is the sharing aspect
<lain> yeah, avoiding deadlocks
<azonenberg> where you have to declare it as being shareable to any queue it might be rendered on
<lain> ohh
<lain> that sharing aspect
<azonenberg> yeah
<lain> hm yeah...
<azonenberg> So you need to have some way for QueueManager to give you a list of any queue that could possibly be used for rendering, and any queue that could possibly be g_vkTransferQueue
<azonenberg> and share it with all of them
<lain> bluh :P
<azonenberg> If we can statically allocate queues on systems with more queues, and eliminate locking, that will be good
<azonenberg> but we can't give up functionality on lower end hardware outright
<lain> or I could have MainWindow pass its render QueueHandle to TextureManager when it instantiates it
<azonenberg> But the render QueueHandle isn't guaranteed to be the same underlying queue object every time you lock it, right?
<lain> it is yes
<azonenberg> ah, ok
<azonenberg> in that case that's likely the simplest solution
<azonenberg> yay concurrency :D
<lain> the only annoying part there is I need to expose a mutex from TextureManager in case multiple Texture objects are being created at the same time (and thus all trying to hit TextureManager::m_cmdBuf at the same time)
<lain> hm
<azonenberg> You'd need one anyway, no?
<lain> yeah
<azonenberg> because TextureManager has the list of textures that isnt thread safe
<lain> ah, true
<azonenberg> But i'm not sure if we actually ever allocate textures outside the main thread
<azonenberg> i think we only ever use textures for tone mapping (in the main thread) and when loading toolbar icons
<lain> I'll just make a note
<azonenberg> Yeah i think it's safe to not lock it
<azonenberg> we do tons of stuff in background threads and on other queues but it's all compute shaders
<lain> hmm
<lain> my branch is a lil crashy, I'll want to fix that before we attempt a merge
<lain> looks like that's my agenda for tomorrow
<azonenberg> ok. i'm working on gui preferences and font handling in ngscopeclient
<azonenberg> shouldn't be conflicting
<lain> I think it's just an order of operations issue with resource release, will confirm tomorrow
<lain> oo, also some fence issues around command buffers, I can see how that would slip by with how I've implemented queue manager.. well, should be a pretty easy fix
bgamari has quit [Quit: ZNC 1.8.2 - https://znc.in]