<azonenberg>
lain, @louis: so i'm starting to think about adding memory traces to scopehal soonish
<azonenberg>
trying to think what the best model for that is
<azonenberg>
in particular, should it be a scope? a filter? something entirely new?
<azonenberg>
we need to be able to serialize the memorized waveform to disk
<azonenberg>
ideally we should be able to memory-ify anything, not just a raw scope trace
<azonenberg>
so things like a filter output
<d1b2>
<azonenberg> (ping @louis since it didnt tag you before)
<azonenberg>
also miek, electronic_eel?
<azonenberg>
and balrog i guess
<azonenberg>
the simplest option is simply to make a memory be a filter that takes a scope channel and a timestamp as inputs, and then magically goes into history and pulls out the older waveform and displays it. but i really don't like that idea
<azonenberg>
because it creates a lot of spaghetti between history (implemented in the frontend) and filters (backend)
<azonenberg>
and it doesnt let you do history of a filter
<electronic_eel>
azonenberg: one thing i often want for history traces is shifting them around in time, so that i can properly compare them to other traces that have slightly different trigger points or similar
<electronic_eel>
when you are able to do math on a history trace and a current trace (like diff them or whatever), then this kind of time shifting would also be very helpful.
<electronic_eel>
when saving the data, do you plan to also store the filter graph, so that you could not just load source data as history trace, but also the results of some filter processing? and even tweak the filter settings after loading?
<electronic_eel>
i think this would be quite a powerful feature
bvernoux has quit [Quit: Leaving]
<azonenberg>
electronic_eel: sooo thats where it gets tricky
<azonenberg>
exactly what we want to snapshot and do things with
<azonenberg>
There is already a "deskew" filter that lets you apply a horizontal shift to a waveform
<azonenberg>
so stacking that with a memory trace would be trivial
<azonenberg>
anyway, so the easiest way to implement it is something similar to how most current scope GUIs do it
<azonenberg>
you create a memory filter as a single-input filter
<azonenberg>
and it has a button that lets you snapshot the input
<azonenberg>
at which point from then on, as the input changes the output stays the same until you hit the button again
<electronic_eel>
but i guess loading the filter graph along with history isn't that easy
<azonenberg>
well you can apply this to the output of a filter is what i'm saying
<azonenberg>
its just another filter, its input can be a channel or a filter
<azonenberg>
in particular though, this does *not* enable you to directly use an old historical waveform as the input to a filter
<azonenberg>
you have to go back to the old timestamp, create/update a memory trace to snapshot the old waveform
<azonenberg>
then move back to the present and do math between the scope channel and the memory trce
<azonenberg>
The filter graph structure is also completely decoupled from history
<azonenberg>
you dont have snapshots of old filter settings etc
<azonenberg>
the filter graph is always the single most up to date copy, all that changes is the contents of the waveforms as you navigate history
<electronic_eel>
ok, so you could save the source data you have and then in the future load that data again. then configure the filter graph as you had it, based on the loaded history waveforms. this would allow you to tweak the filter settings in the future
<electronic_eel>
but the filter settings wouldn't be automatically saved together with the waveform and be seamlessly loaded together with the source data
<azonenberg>
Correct
<azonenberg>
This is intentional, as the typical use case is to configure the filter graph as you go, set up decodes and such
<azonenberg>
occasionally tweak parameter slightly, maybe add a de-embed or something
<azonenberg>
then as you go back in time you're seeing the current analysis applied to the previous data
<azonenberg>
if you want to snapshot the old filter graph you'd have to save a separate scopesession of just that
<electronic_eel>
i'd say this would enable all the important analysis you want to do, but sometimes you'd need a few more clicks in the gui. not a showstopper
<electronic_eel>
saving history traces should be fast and convenient though
<azonenberg>
Yeah
<azonenberg>
So in ngscopeclient my thoguht is that you'd simply right click the channel button and click "snapshot"
<electronic_eel>
i really hate it when you want to save in quick succession and have to fiddle in a unique filename with a scrollwheel first...
<azonenberg>
and then it would create a memory filter showing the content of that channel at that point in time
<azonenberg>
if at some point in the future you wanted to update the snapshot with new state you'd simply open filter properties of the snapshot
<azonenberg>
and there would be a button to make it update
<azonenberg>
you could also change the input to make it snapshot a different waveform, etc
<azonenberg>
basically it would function like a DFF for waveforms that clocks on creation plus whenever you click the button
<azonenberg>
and you can rewire the input between clocks as you see fit
<electronic_eel>
the snapshot is part of the scope session, correct? would it be possible to load history traces from other scope session files?
<azonenberg>
Not directly, but you could export a waveform from another session to an interchange format like CSV and then load that
<electronic_eel>
hmm, i think directly being able to load it would be a good idea
<azonenberg>
So basically the more you reach laterally the messier things get
<azonenberg>
for example, a snapshot function is much cleaner than being able to directly reach into history and pull out "the waveform from yesterday's channel 1 of scope foo"
<azonenberg>
even if you can replicate that simply by jumping back to yesterday in history, snapshotting channel 1, and moving back to the present
<azonenberg>
if you enable import of another scopesession users will probably start to expect that you be able to load old filter values from the scopesession's filter graph, etc
<azonenberg>
it's quickly going to get very very messy
<azonenberg>
One *possibly* useful feature would be importing an existing scopesession and adding it to the current project
<azonenberg>
but there's a lot of corner cases there
<azonenberg>
for example, what happens if a scope in the current project was already in the old session
<azonenberg>
do you load those waveforms and add them to history of the current scope?
<azonenberg>
do you make a new scope?
<azonenberg>
what if the two sessions overlap in history? say you saved session A, moved a bunch of stuff around, take two waveforms in session B
<azonenberg>
reconnect to A, take two more waveforms
<azonenberg>
then merge them
<electronic_eel>
that session add seems messy to me, since you would add a lot of stuff to the current session that you might not need, but would still take resources and could create conflicts
<azonenberg>
how does this play with the filter graph, given that the two sessions likely will not include waveforms from the same timestamps? do you have half the filters go blank then the other half?
<azonenberg>
exactly
<azonenberg>
So you're proposing being able to reach directly into a scopesession and *only* load the waveform data from a single channel at a single timestamp?
<azonenberg>
(keeping in mind that since we lazily evaluate filters, the serialized data is only going to be raw scope channels and not any filters)
<electronic_eel>
i thought more of something similar to importing a csv. but just that you select a scope session and you then can select a channel from that session
<azonenberg>
well what i mean is, how does this play with history and such given that a filter has no concept of history
<azonenberg>
you'd have to select a channel and timestamp
<azonenberg>
not that this can't be done
<electronic_eel>
that is why i wrote above that shifting the history traces in time is important
<azonenberg>
when you say "shift in time" do you mean shifting the timestamp metadata, or making a slight left/right tweak to comepnsate for skew or trigger setup changes?
<electronic_eel>
the way i ususally use history traces is to compare my current traces to some older traces
<electronic_eel>
so it doesn't help if the older trace has a timestamp from yesterday and it doesn't align with the current one
<azonenberg>
So you can have waveforms with different absolute timestamps displayed simultaneously
<azonenberg>
e.g. a CSV import will have a timestamp of the file's modification time
<azonenberg>
This does get fun when markers are involved
<azonenberg>
but you can totally do math between waveforms that have different timestamps
<azonenberg>
the trigger phase is where it gets messier, not every filter can do the resampling needed to subtract waveforms that have different start points
<azonenberg>
to work with*
Fridtjof has quit [*.net *.split]
<electronic_eel>
how about comparing them not with math, but visually? can i easily drag them along on the screen until they line up in a way i think makes sense?
<azonenberg>
so there's two different attributes
<azonenberg>
there's the absolute timestamp which is mostly ignored by filters
<azonenberg>
and there's the trigger phase which is basically the x axis position of sample #0
<azonenberg>
the deskew filter changes that
<azonenberg>
right now you have to type a skew in by hand but it wouldn't be implausible to add a gui feature for that
Fridtjof has joined #scopehal
<azonenberg>
most scopes support deskew in hardware but some cheaper ones don't, the primary intention is to allow you to ex post facto correct for probe skew if it wasn't done when the waveform was captured
<azonenberg>
but you can make the shift arbitrarily large for whatever reason
<electronic_eel>
there could be a filter that finds the skew value that minimizes the differences between two traces, that could be used to automatically lign up a current and history trace in many cases
<azonenberg>
hmmmm
<azonenberg>
so we actually have such a thing already, sort of, that is used for the multiscope calibration
<azonenberg>
the issue is that it's *very* expensive
<azonenberg>
so for two waveforms of lengths N and M, with a max search space of +/- P samples from the default (no shift) alignment
<azonenberg>
it's O(N*M*P)
<electronic_eel>
yeah, you don't want that running all the time
<azonenberg>
maybe we should create a new class of filter "triggered" or "run on request" or something
<azonenberg>
filters that only execute when explicitly requested
<azonenberg>
so memory would be an example of that
<azonenberg>
auto align would too
<electronic_eel>
i like that idea
<azonenberg>
so basically the normal refresh action would do nothing
<azonenberg>
the filter properties dialog would display a button
<azonenberg>
and you could click it to execute the action
<electronic_eel>
would that run on request be a fixed property for some filters or would it be an option for all filters, with sensible default values?
<azonenberg>
So i think it would actually be orthogonal to the regular filter thing
<azonenberg>
you could put anything you want in the rfresh method
<azonenberg>
then an ActionFilter would expose a vector<string> of actions you can perform
<azonenberg>
to, say, auto train an equalizer
<azonenberg>
each action name will be displayed as a button in the gui
<azonenberg>
when clicked it calls ActionFilter::OnAction("foo")
<azonenberg>
actually we'd probably make it an enum with names you can query for the integer values
<azonenberg>
but anyway
<electronic_eel>
if you wanted the auto align filter to continously run and follow your waveform, even if it is expensive, you couldn't do that when it exposes just this action string/enum, right?
<azonenberg>
Correct
<azonenberg>
but with any reasonable memory depth it takes a LONG time
<azonenberg>
gpu accelerating will help but still
<electronic_eel>
wouldn't that design be a bit limiting the options? i mean there could be a auto align filter that does the full comparison just once and then just searches a few µs back and forth to adjust for any trigger jitter or similar
<azonenberg>
So then you'd have the full align be the triggered action and then the small window search be in Refresh()
<azonenberg>
this does not remove the refresh method, it's orthogonal
<azonenberg>
the intention is to have slow actions, or those that you explicitly only want to run here and there to initialize something, be triggered on cue
<azonenberg>
and then the filter graph otherwise runs like normal
<electronic_eel>
ok, but this has to be decided by the developer writing the filter code, not the user clicking some options in the dialog
<azonenberg>
Correct
<azonenberg>
the way the filter graph scheduler works, making the main refresh path triggered gets messier. also this model allows arbitrarily many actions, not just one, per filter
<electronic_eel>
it could also be the other way around: the user has far too many filters configured for the cpu/gpu available. to still be able to work, they don't want continous updates of the filters, but just want them to be run on click
<azonenberg>
That's why graph updates run in a background thread