<azonenberg>
Oh, WSL is a full VM? I thought it was a WINE-type layer and did not actually run a full Linux kernel internally
<azonenberg>
just a compatibility layer to translate linux syscalls to windows equivalents
<d1b2>
<johnsel> WSL2 is actually a full linux kernel, that runs on the same level as the NT kernel, on the hyper-v(isor) and with some pretty nifty integrations even like gpu sharing to the linux kernel w/ acceleration support and a built in X server on the Windows end that lets you natively present linux windows as if they are Windows windows.
<d1b2>
<johnsel> It would be an option to use directly if usb device support wasn’t a dragon of an implementation
<azonenberg>
yeah i mean we want to have native windows support in general
<azonenberg>
as far as nested virtualization goes, apparently xen does support it but it can be somewhat experimental
<azonenberg>
unsure how well it will work
<azonenberg>
i was assuming you would be running natively on the windows instance vs through WSL
<azonenberg>
If we switched CI platforms and ran the windows build natively on a windows instance, without using WSL, it should be fine right?
<azonenberg>
We just couldn't use the github runner/platform for it?
<d1b2>
<johnsel> I would do that too, the runner doesn’t though
<d1b2>
<johnsel> The runner has a hard dependency on wsl it seems
<azonenberg>
I see
<azonenberg>
What i meant was, if we hypothetically switched to, say, CDash
<azonenberg>
or any other CI platform besides github actions
<d1b2>
<johnsel> Potentially an option, but most ‘simple to use’ services expect the ci process to run in docker containers
<d1b2>
<johnsel> I have to do some more digging still before I have a good picture what gives the least effort path to something that works well
<azonenberg>
Ok
<azonenberg>
and yeah cdash is just a dashboard
<azonenberg>
you have to pull the code, build it, etc yourself
<azonenberg>
it just is a place to dump results
<d1b2>
<johnsel> All build automation platforms and techniques come with their presupposed use cases and environments
<azonenberg>
in the past iirc i had a cron job that would pull and upload at regular intervals on my actual workstation
<azonenberg>
(this was for a private repo only i was working on )
<d1b2>
<johnsel> We have a pretty big matrix of requirements with all the platforms and the msys2 is a little extra funky on top
<azonenberg>
Yeah agreed
<azonenberg>
if it was an easy problem i wouldn't have someone else looking into it :)
<azonenberg>
i'd just do it myself lol
<azonenberg>
anyway, keep us posted with what you come up with
<d1b2>
<xabean> WSL1 was more of a "lets do on the fly syscall emulation" WSL2 is full blown VM
<azonenberg>
ah i see
<azonenberg>
so its not really a windows subsystem anymore
<azonenberg>
in the literal sense
<d1b2>
<xabean> I think microsoft just really didn't want to spend the time reinventing the wheel
<d1b2>
<johnsel> It is a subsystem in the sense that there’s a fully integrated linux subsystem in your windows os.
<azonenberg>
yeah but not in the sense of the os/2 subsystem or posix environment subsystem
<azonenberg>
or the win32 subsystem
<d1b2>
<johnsel> I’m not sure where you draw the distinction, it’s not a vm if that is what you mean
<azonenberg>
The term "subsystem" had a specific meaning in windows NT architecture
<azonenberg>
and it appears WSL1 met that, while WSL2 does not
<d1b2>
<johnsel> Ah yes, I see what you mean now, though in reality none of those subsystems function like that anymore with the changes and introductions made over the years. This is all replacements of the once posix subsystem in lineage, but there is a full linux kernel that talks to the hypervisor, with drivers for that and then both userlands are linked on the presentation stuff with wslG.
<azonenberg>
well most no longer exist
<azonenberg>
i dont think either the posix or os/2 are present in modern win11
<azonenberg>
or even since XP?
Degi_ has joined #scopehal
<d1b2>
<johnsel> Posix became unix subsystem then wsl and now wsl2, os/2 is long gone, win32 is also scrambled over the years introducing kernel level security shrinking the usermode win32 studf and the hypervisor once again pulled a lot away from the kernel is my understanding
Degi has quit [Ping timeout: 256 seconds]
Degi_ is now known as Degi
<d1b2>
<johnsel> Though admittedly I was more a sysadmin guy and left both windows and that work sometime around win 7/8
<d1b2>
<johnsel> Double checking myself it looks like they used to be well defined terms for specific things but nowadays those concepts have morphed into similar in functionality but technically unrelated indeed.
<d1b2>
<johnsel> And if it was easy everyone would be doing it.
<d1b2>
<johnsel> And I will
<d1b2>
<johnsel> Might take a few days before I have something concrete but I’ll keep spending a few hours here and there on it
<azonenberg>
Sounds good
<azonenberg>
I guess hold off on the other CI stuff we talked about, like logging build failures to the chat, until then
<azonenberg>
since we might have to change whatever that system is if we switch CI platforms
<d1b2>
<johnsel> Yeah that was my thought too
<d1b2>
<johnsel> Nobody wants to see 50 ‘testing ci, plz work’ messages anyway
<d1b2>
<johnsel> Not that I would want to make the change, but why not gitlab btw for this project? Many FOSS projects I have seen hosted there
<azonenberg>
Because we started out on github and havent had a reason to move?
<d1b2>
<johnsel> Haha, fair enough.
<d1b2>
<johnsel> You are aware that GitHub is considered very foss unfriendly by certain foss ‘rating orgs’? Just to give context to my question. I personally don’t find it such a big deal
<azonenberg>
I have not seen any issues. They're bound by US law, so if you're associated with e.g. a sanctioned organization in .ru you'll have problems
<azonenberg>
or if you're posting leaked info somebody else has a copyright on you can expect to be DMCA'd
<azonenberg>
but that's inherent to any US hosted service
<azonenberg>
or most of EU these days too
<d1b2>
<johnsel> Oh no it predates all that. Let me dig it up
<azonenberg>
neither is really a problem for our use case
<d1b2>
<johnsel> This is not the original article but these are the subjects that they discussed
<d1b2>
<johnsel> So more the commercial vs foss stakes colliding, not political issues
<azonenberg>
Yeah. well, scopehal is permissively licensed and expressly intended to be friendly to commercial users etc
<azonenberg>
and sure the site's code isn't public, but that isnt really a huge issue
<d1b2>
<johnsel> Sure but do you like their ai scooping up your filter code and selling them in a subscription
<d1b2>
<johnsel> That kinda stuff
<azonenberg>
I wrote the code to solve a problem i have
<azonenberg>
somebody else using it for any purpose doesn't impair my ability to get work done
<azonenberg>
i'm mildly annoyed at them doing it without proper credit, but that's much more of an issue for copyleft projects
<azonenberg>
i expect this to be litigated at some point and in the meantime am ignoring the issue
<d1b2>
<johnsel> No and that’s fine, lots of foss believers are more stringent on it though
<azonenberg>
sooner or later someone, likely a big GNU project, is going to take microsoft to court over this once they find a non-GPL project using copilot generated code they can prove was lifted verbatim from say the linux kernel
<azonenberg>
and whatever happens happens
<d1b2>
<johnsel> So in the context of ci it might have made sense to look at e.g. gitlab too
<azonenberg>
And i've had the permissive vs copyleft discussion with RMS to his face
<azonenberg>
my views are quite clear: my code is a gift to the world and i intend it to come with as few strings attached as reasonably practical to maximize its utility
<azonenberg>
i'm not fighting an ideological war against evil proprietary software
<azonenberg>
i wrote it for my own purposes, i lose nothing by letting others use it
<azonenberg>
and if they share improvements i might find useful - which they don't have to, but are encouraged - I benefit
<azonenberg>
so from my perspective there is a potentially significant upside and no real downside
<azonenberg>
e.g. a GPL purist would hate the idea of my filter code ending up on a commercially sold scope unless that scope's firmware was open source
<azonenberg>
while from my perspective, if that happens
<azonenberg>
game over and i won :p
<d1b2>
<johnsel> Yeah I share that view, as long as it has fulfilled my purposes I am happy to share it with the world to do with whatever they want. Assuming the code itself won’t give you a migraine and make me blush
<d1b2>
<johnsel> And that I don’t have other stakeholders that might think differently
<d1b2>
<johnsel> Why game over though?
<azonenberg>
Exactly
<azonenberg>
Well, because it means i'm no longer playing catch-up with scope vendors
<d1b2>
<johnsel> You accomplished your life’s goal?
<d1b2>
<johnsel> Or current at least
<azonenberg>
if they're copying from us and not the other way around, we're now playing on a level field
<d1b2>
<johnsel> Yeah that makes sense
<azonenberg>
this is why i'm not afraid of people forking our code and makign a commercial product from it
<azonenberg>
that product will cost money, but may have features we lack
<azonenberg>
and as far as i'm concerned, that's a fair free-market move
<azonenberg>
end users are free to choose the more featureful version, or the one they can study and extend
<azonenberg>
depending on how much they value access to the code vs whatever that company charges
<azonenberg>
If people think the features they added are worth paying for, they're welcome to charge people for that
<azonenberg>
copyleft policy would consider the idea of a closed fork of an open product immoral, i say it's a valid strategy but by closing your fork you're making it less desirable
<azonenberg>
so if (value of added features - value of source access) > sale price
<azonenberg>
people will buy it
<azonenberg>
i suspect that would be hard to do when competing with "free"
<d1b2>
<johnsel> Will you still keep that view if a billion dollar company forks your product and puts in the capital and manhours to leave you in the rear view mirror and you never see a dime?
<azonenberg>
Yes. Although it's not going to happen
<azonenberg>
So that's the other reason
<azonenberg>
there's about five companies globally who might have the motivation to do that
<azonenberg>
all of them have huge amounts of money sunk into their existing codebas
<azonenberg>
many of them have their own competing PC application (e.g. MAUI Studio from LeCroy) that is $$$$ and only works with thier scopes
<azonenberg>
some vendors even charge you separately for software options for offline analysis vs what's on the scope
<azonenberg>
and vendor lockin is key to their business model to some extent
<azonenberg>
one of our key value propositions here with glscopeclient is that we can work with any scope, made by anyone, and you dont have to pay for any of the software options
<d1b2>
<johnsel> Yeah there’s the real value
<azonenberg>
so no scope vendor would ever make a commercial fork of it as it would be self defeating
<azonenberg>
i suppose it's possible a third party company could try to do so, but they would have to compete with free
<azonenberg>
i think it's unliekly they'd get far
<azonenberg>
much of the interest i've had from commercial end users has been based around the net TCO reduction
<azonenberg>
even if glscopeclient currently lacks a feature they want
<azonenberg>
it's cheaper to pay an engineer to write it once and have it forever
<azonenberg>
vs paying license fees for every scope in perpetuity
<d1b2>
<johnsel> Yeah they really are golden eggs
<azonenberg>
So anyway, given how we fit into the overall T&M ecosystem
<d1b2>
<johnsel> Or rather the chicken even
<azonenberg>
i simply don't see a commercial fork as being a threat to the project
<azonenberg>
That said, it's entirely plausible someone could have a business of selling plugins with decodes for specific protocols or something as closed source modules
<azonenberg>
we have a binary plugin model in part to allow that
<azonenberg>
it wouldn't hurt us at all
<azonenberg>
if we later made an open decode for that same protocol it would compete as free vs their potentially more advanced feature set
<azonenberg>
and again, people can decide if they want to pay for it or not
<azonenberg>
but we don't lose out
<azonenberg>
in fact, the open nature of the project and decodes has been a key value proposition
<d1b2>
<johnsel> Yes I think you’re right
<azonenberg>
louis told me a while back about the time that he needed to debug a device implementing OneWire with slightly out of spec timings
<azonenberg>
our decode refused to parse it, but in a matter of minutes he made the error checking tolerances looser, recompiled, and was able to parse it
<azonenberg>
with a blob decode you'd be out of luck
<azonenberg>
so essentially the open nature of the project is inherent to what it is, and making a closed fork would make it vastly less desirable to end users
<d1b2>
<johnsel> Yeah I think I said that before already that is what initially got my interest, together with the performance
<azonenberg>
Yep
<d1b2>
<johnsel> Alright I understand your views a little bit better on the project now
<azonenberg>
Sooo, potential architectural limitation to be aware of
<azonenberg>
my RTX 2080 Ti appears to have a max of 4GB max memory allocation
<azonenberg>
and max compute group count in the X axis is 2^31 - 1
<azonenberg>
which is to say, going beyond about one gigapoint of memory depth within a single channel is potentially problematic
<azonenberg>
we we would need to either make our shader invocation 2-dimensional, or split memory allocations into several, or both
<azonenberg>
i don't *think* this is a huge deal because you're going to run into video memory etc limitations too with that
<azonenberg>
if your waveforms are dense packed you will need a float32 per sample, so with 1G points that's 4GB of memory for a single channel
<azonenberg>
and obviously when you start doing a filter graph on that gigapoint sized waveform, now you need enough memory to store the input and output
<azonenberg>
you'll run out very fast on a card with a few gigs of memory
<azonenberg>
But this is potentially something to think about for folks who might be working with stupidly large waveforms
<azonenberg>
It looks like you can get the RTX 3090 Ti with up to 24GB of GDDR6x
<_whitenotifier-7>
[scopehal] azonenberg fb9c1b3 - Initial version of SubtractFilter is now working using Vulkan! Need a lot of refactoring before we can reuse this code elsewhere, but it's alive.
<azonenberg>
I went from like 8 to 3.4 GB of video ram used by closing a few kicad instances
<azonenberg>
i had trouble opening my half gigapoint capture from oxide until i did that
<azonenberg>
maybe it's time for me to get a beefier GPU... I could move this 2080 to the microscope workstation which currently has a 2014 vintage quadro in it
<azonenberg>
and get something with more vram for the main box
<azonenberg>
(as it stands i can open the giant capture and subtract P from N, but i don't know if i would have enough memory left over to run any kind of postprocessing on the filter graph etc)
<azonenberg>
admittedly 2x 512M point captures is quite large
nelgau has quit [Remote host closed the connection]
<_whitenotifier-7>
[scopehal] azonenberg 02db420 - TRCImportFilter: refactoring in preparation for GPU acceleration
<_whitenotifier-7>
[scopehal] azonenberg 54f84e7 - Added support for push constants to ComputePipeline. Implemented Convert16BitSamples as a Vulkan shader.
<azonenberg>
i think next up i'll try GPU accelerating waveform sample processing
<azonenberg>
i.e. converting the raw adc codes to fp32 samples
<azonenberg>
at that point at least for the lecroy driver we will have ADC codes -> GPU memory and hopefully be able to pass that GPU pointer directly to other kernels and all the way to the eventual Vulkan rendering shader
<azonenberg>
without ever having to copy it back to the CPU
<bvernoux>
Unfortunately everything is closed source about that ones
<bvernoux>
even NanoVNA-Qt which seems based on the old one for V2
<bvernoux>
Whith no support evolution since 2020 ...
massi has quit [Remote host closed the connection]
<bvernoux>
They are using exactly same CPU as before on NanoVNA V2 Plus 4 too => GD32F303
<bvernoux>
Seen in the firmware update for the V3 (VNA6000)
<bvernoux>
So pretty disappointed as it is pretty and UI is ugly
<bvernoux>
Only 1024pts also with embedded UI
<bvernoux>
for a pro VNA I always use 1601pts for accurate measurements...
<azonenberg>
bvernoux: johnsel is working on CI stuff
<azonenberg>
he's got a long list of pending fixes and this is just one
<miek>
are there any plans to do another run of AKL-PT5 EVT units?
<azonenberg>
miek: No. Those engineering samples were enough to get initial user feedback. I did a few more tweaks since the EVT2 round and pushed the bandwidth from ~5.3 GHz to over 8.5 with a different tip resistor, but that added some high freq peaking
<azonenberg>
the latest prototype is flat like +/- 0.75 dB from DC to 7.3 GHz, then dips down a bit (but not quite to -3 dB) by 8.3 GHz
<azonenberg>
then it goes off scale on my 8.5 GHz VNA, but judging by time domain response peaks again. That peak is problematic and causes overshoot, so i'm playing with filters to remove it
<azonenberg>
i have a new filter from minicircuits coming tomorrow that should take care of that quite nicely
<azonenberg>
assuming i have satisfactory response on that version, the plan is to work with MP on tooling up for production
<azonenberg>
there will likely be a 20ish piece PVT run using actual production infrastructure, tooling, panelized PCBs, etc
<azonenberg>
I will not be listing the PVT for public sale due to limited quantity (I don't want to be paying for a digikey listing that's out of stock almost immediately and just gets people's hopes up for no reason when none are available)
<azonenberg>
So the plan is to hold onto most of the PVT units for internal use, although a few will be available for private sale on request to folks in this channel etc.
<azonenberg>
And assuming PVT goes well we'll place a volume order for several hundred tip resistors
<azonenberg>
once those arrive, we'll set up listings at distributors and stuff boards in small batches as orders come in
<azonenberg>
The other wrinkle i'm dealing with is what looks like some potential process issues in the overmolding/potting compound on the HMLs that i'm talking to vishay about, sending a few parts back for FA
<Johnsel>
bvernoux: yes that fixes building on Windows in general. It does not fix building on Windows on the CI though. Also, check out liteVNA for a much cheaper 6ghz vna. Hugen is one of the early cloners
<Johnsel>
I have a comparison picture of it vs the R&S ZVB8 scanning a 2.4ghz filter from 2-3ghz, the response looks near identical from 2.25 to 2.7
<Johnsel>
ofcourse that's in that chosen example only, but it's definitely not a cheap clone with according results
<Johnsel>
azonenberg: you might want to think about a streaming processing architecture to limit memory use?
<azonenberg>
Johnsel: that doesnt work for the use case of a scope scrolling through a large waveform
<azonenberg>
there's no real practical way to not store the whole buffer
<azonenberg>
when you're dealing with gigapoints of sample data you need lots of ram and there's not really any way around that
<azonenberg>
streaming works for the SDR use case
<Johnsel>
RAM, yes
<azonenberg>
not so much here
<Johnsel>
vram, not necessarily
<azonenberg>
well the whole point of glscopeclient is to keep things as much as psosible on the GPU so we can hardware accelerate computation and rendering
<azonenberg>
if waveform data lives on the CPU, as you scroll around the waveform you're constantly pushing more data to the card
<azonenberg>
if it's on the GPU all you have to do is update the view transform and re-render
<azonenberg>
History will be kept in ram, and paged out to disk cache if possible
<azonenberg>
but the active set of waveforms on screen has to fit in vram
<azonenberg>
i dont think this is a huge problem. 99.9% of users won't be working with e8 point waveforms on a regular basis
<azonenberg>
most scopes dont even go that high
<azonenberg>
iirc my lecroys max out at 64M on all channels, or the waverunner can do 128M on 2 channels
<azonenberg>
the capture giving me trouble is 512M
<Johnsel>
I was more considering multiple filters layered on top of each other etc
<azonenberg>
Sooo there is room in the architecture to tweak this
<azonenberg>
in particular, it's required that everything be GPU readable
<azonenberg>
but it does not have to actually live on the GPU, pinned memory is an option too
<azonenberg>
there are hints in the AcceleratorBuffer class you can tune to say how often you expect to use the data where
<Johnsel>
right just keeping some potential future growth path in the back of mind
<azonenberg>
and it will choose a memory mapped file, non-pinned host ram, pinned host ram, on-device memory, or a mirror of pinned host ram and on-device memory
<Johnsel>
PXL will also bring weird new possibilities
<azonenberg>
What i'm getting at is, we have the option of fine tuning storage even on a per filter instance basis in the fututre
<azonenberg>
And even move storage dynamically
<azonenberg>
say using vram during evaluation of a filter that is never rendered
<azonenberg>
but once that processing completes, using it as an input to another filter
<azonenberg>
and then freeing that vram
<azonenberg>
AcceleratorBuffer can do this already
<Johnsel>
CXL*
<azonenberg>
we just need the glue to tell it when to reallocate and move data
<azonenberg>
So we have a ton of flexibility, we just need to figure out how to expose the low level details via the gui
<azonenberg>
it's not a trivial problem and it's not one i think we need to solve immediately
<azonenberg>
because yes, if you have a chain of a lot of filters on a gigapoint waveform you will need a ton of vram with things the way it is
<Johnsel>
not trivial at all, but it is a potential place to exceed other scopes in functionality
<azonenberg>
i mean gpu acceleration in general is something no other scope can do
<azonenberg>
And at the moment i'm generally optimizing for throughput even if this puts some limits on total waveform size
<azonenberg>
Which i dont think is unreasonable. if someone is working with gigapoint waveforms they probably have a very big scope - the dataset in question came from a LeCroy LabMaster that costs as much as a small house
<azonenberg>
and asking them to buy an RTX 3090 with 24GB of VRAM to process that data doesn't seem excessive
<azonenberg>
an extra $1K is nothing to a company like that
<azonenberg>
and the hobbyists and small businesses using siglents with 10M points max memory will have no trouble with basically any gpu on the market
<Johnsel>
Sure but you are streaming in data anyway, GPUs and their APIs are also built with streaming data (and variable detail of the data) in the back of the mind because that is what games do anyway
<azonenberg>
Yes. What i'm saying is, if you want to do that you can always set the scope memory depth to be less
<azonenberg>
and stream more waveforms per second
<azonenberg>
you have control over the block size
<azonenberg>
it's just that there's dead time in hardware between the blocks in some cases
<azonenberg>
(if you are using sequence mode capture, that dead time can be very short - we support this in at least the LeCroy driver but you have to set it up on the scope as libscopehal currently lacks APIs to control sequenced memory)
<azonenberg>
it understands how to use it if you enable it
<azonenberg>
and in that case, each of the sub-waveforms of the sequence is logically a separate waveform as far as libscopehal is concerned
<azonenberg>
and only one of those has to fit in vram at a time
<Johnsel>
yes you can do it this way too, but you're pushing those implementation details to a higher level essentially. If your gpu and filter see all data at the highest detail it allows you to do things like software defined triggering
<Johnsel>
I'm not arguing that you should do it my way btw, just that that is what I had in the back of my min
<d1b2>
<xabean> I wonder how viable it would be for a crap video card for display of waveform data, and run the compute bits on one of those fancy headless videocards -- e.g. for those folks who have those large scopes that cost the price of a single family home.
<d1b2>
<zyp> I'm not sure how the last part relates, but using multiple devices in vulkan shouldn't be any issue at all
<Johnsel>
that's actually a pretty interesting question, I don't think vulkan compute can be initialized on headless seperate from the presentation stuff
<azonenberg>
So, right now we default to using the first supported Vulkan device for compute
<d1b2>
<zyp> yes it can
<azonenberg>
Whatever it may be
<azonenberg>
there will eventually be preferences for configuring that
<Johnsel>
Vulkan supports headless rendering?
<d1b2>
<zyp> in vulkan you have to pick devices explicitly and there should be no issue opening several at a time
<azonenberg>
for the next week or so at least, the output of all filters - whether in software or vulkan or opencl - gets copied to CPU-side memory at the end of processing
<azonenberg>
and then gets pushed to the renderer, which - for the near term - is an OpenGL compute shader
<azonenberg>
the renderer then outputs a grayscale fp32 texture for each waveform
<azonenberg>
this then gets passed to the final compositor, an OpenGL vertex/fragment shader, which maps the grayscale to RGB values
<azonenberg>
and alpha blends it with software rendered (Cairo) over/underlays like cursors, protocol decode event bubbles, etc
<azonenberg>
moving forward, the waveform -> fp32 pixel will be transitioned to Vulkan
<azonenberg>
and the final compositor will remain OpenGL
<azonenberg>
all of the Vulkan stuff at this point is still headless and does not need to be the same GPU as the compositor, which is the only thing that has to be your actual display
<azonenberg>
in fact, all of our planned Vulkan for the near to mid term is compute. no actual rasterization at all
<azonenberg>
(waveform ->fp32 pixels is actually a software rasterizer in a compute shader optimized specifically for waveform drawing)
<azonenberg>
this is how the opengl renderer does it now, we'll be using the same shader and just porting to Vulkan
<azonenberg>
Anyway, very long term (2023-2024) when gtk has better vulkan integration, we will probably transition the compositor to Vulkan and stop using GL entirely
<azonenberg>
at that point, we may or may not decide to support cross device operation where the compositor and compute/rasterization run on separate devices
<azonenberg>
From now until then, it will work out of the box. the compositor will use whatever GPU you are running your display on
<azonenberg>
and compute will use whatever GPU you selected for vulkan acceleration
<azonenberg>
right now what we do is look through the list of suppoted devices, pick the first discrete GPU
<azonenberg>
if none found, pick first integrated CPU
<azonenberg>
if none found, pick a software emulation layer like llvmpipe/swiftshader
<azonenberg>
first integrated GPU*
<azonenberg>
but there will eventually be a preference to force use of a specific device
<azonenberg>
Another potential use case - will require more code refactoring and design to support
<azonenberg>
is simultaneous use of multiple GPUs for different parts of a filter graph
<azonenberg>
the challenge being if you have dependencies between the different parts, you need to copy from card to card
<azonenberg>
But these are all things that could potentially be added in the future if we had a reason to
<azonenberg>
in general, vulkan gives us a lot more control about exactly what resources we use for a given task
<electronic_eel>
is there an easy way in the current glscopeclient code to test if the vulkan stuff works as expected? like if the gpu drivers are correctly loaded and so on. i want to give it a try with my amd gpu and it's open source drivers
<Johnsel>
oh that's a pretty non standard way to use vulkan, good to know
<Johnsel>
I'll probably have to look deeper into it then, in fact you can see the CI as this exact case of a headless render
<azonenberg>
Johnsel: well we're using it entirely for compute acceleration for now
<azonenberg>
our current waveform rendering, like i said, is a special case of compute
<azonenberg>
electronic_eel: just run glscopeclient latest from git. when it starts up you should see it log vulkan initialization
<azonenberg>
if you --debug
<Johnsel>
sure but just the fact that both gl + vulkan get initialized might do weird stuff
<azonenberg>
the subtract and LeCroy .trc file import filters are currently accelerated using vulkan
<azonenberg>
Johnsel: oh. yes, that is a bit uncommon but i dont think it's that rare WRT porting complex system
<Johnsel>
that's a lot of driver and API paths being exercised
<azonenberg>
and we're not sharing buffers between them
<azonenberg>
which is where most of the dragons are
<azonenberg>
they're two completely separate pipelines that happen to be in the same process
<electronic_eel>
azonenberg: ah, ok, so just running with --debug should be enough. thanks. will report if i run into any issues
<Johnsel>
yes, but headless has it's own dragons
<azonenberg>
well
<azonenberg>
vulkan headless vs normal is not really any different
<azonenberg>
we just never created a surface to run a graphics pipeline on
<azonenberg>
and only run compute pipelines
<Johnsel>
no but that is precisely why it is a dragon, it expects to be able to run a full graphics pipeline
<azonenberg>
all of the init and compute is exactly the same, we just didn't use some of the other APIs to put stuff on screen
<d1b2>
<zyp> I don't think it's as uncommon as you make it sound, much of the point of vulkan is to give you this sort of flexibility
<d1b2>
<zyp> and no, vulkan by default absolutely does not expect to run a full graphics pipeline, you have to explicitly request one if that's what you want
<d1b2>
<zyp> that sort of assumptions is what vulkan got rid of
<azonenberg>
Exaclt
<azonenberg>
Exactly
<azonenberg>
anyway, the bigger dragon is that right now we have opengl, opencl, and vulkan all coexisting in one app
<azonenberg>
we'll at least be getting rid of the CL soonish
<d1b2>
<zyp> I'm looking forward to try it out on macos once that's possible
<azonenberg>
Get in line, lol. we have quite a few folks excited for it
<d1b2>
<zyp> 🙂
<azonenberg>
Lain is going to be getting started soonish, we're in the process of negotiating the contract and such (MP's company is funding the dev work for the port)
<Johnsel>
I'm not sure, but that is exactly what I called uncommon about it opengl + vulkan (and the headless and MinGW on top of that)
<azonenberg>
in the meantime i'm bringing up vulkan compute and working on transitioning all of the opencl filters to vulkan
<azonenberg>
as well as transitioning away from clFFT and FFTS
<azonenberg>
that will be most of my dev work for the next few weeks. at some point during all of that i'll also be adding the "page historical waveforms out to mapped files" switch
<azonenberg>
which is a pretty trivial one-line method call on the waveform object, i just need to make sure i correctly swap it from file to pinned memory and back at the right times
<azonenberg>
and i might refactor the history window as part of that
<electronic_eel>
so when you get rid of FFTS, will vulkan then become a hard requirement? or is there some software vulkan layer that could take over, of course slower?
<azonenberg>
so the net should be that we can run on even more platforms than before
<azonenberg>
The long term plan is to remove the OpenGL 4.x dependency as well
<azonenberg>
once the renderer is ported to Vulkan we will only require OpenGL 2.x for the compositor
<electronic_eel>
yeah, that would be nice, because vms without gpu passthrough
<azonenberg>
Yeah. Ultimately the goal of this transition is that we should be able to run on more platforms than before
<azonenberg>
namely vms via llvmpipe or swiftshader, and apple via moltenvk
<azonenberg>
as well as raspberry pi 4 which supports vulkan but not gl 4.3
<azonenberg>
As well as moving away from both FFTS and clFFT, both of which are essentially dead (last commits in 2017 for both)
<azonenberg>
and improving FFT performance on large waveforms because clFFT has a hard coded max FFT size of 16M points
<azonenberg>
beyond that, we currently fall back to FFTS
<electronic_eel>
I really want to avoid having to use closed source gpu drivers. I don't have any experience with vulkan yet, so I don't know if it is possible with my amd gpu. i've done some short web search and it looks like it is possible. so i'm going to try glscopeclient with that, likely tomorrow or so
<azonenberg>
here's an example of what i'm seeing on some of the problematic parts
<azonenberg>
it looks like the black overmolding plastic was not properly applied, or was damaged somehow
<azonenberg>
and as a result it's not adequately reinforcing the joint where the lead meets the body of the component
<azonenberg>
and in at least one case i observed a failure where the lead snapped off right at that point
<azonenberg>
Vishay is taking it seriously and investigating on their end. In the meantime i can continue R&D but we're holding off on ordering higher volumes until we have a root cause and are confident the problem is addressed
<azonenberg>
I also have at least one showing a similar failure that I did not send back, which I am going to cross section in my own lab
<azonenberg>
And of course share the resulting photos with them
<d1b2>
<xabean> well that's incredible customer support
<d1b2>
<xabean> afterall since you had to special order them
<azonenberg>
i mean we did also tell them we were gonna be buying a lot more in the future if things went well :p
<azonenberg>
and i'm honestly not super surprised there were some issues because these parts are so obscure
<azonenberg>
they arent going to have as tightly controlled, stable of a process as something they make millions of on a daily basis
<azonenberg>
a bog-standard 1% 0402 i'd be extremely surprised if there were major yield issues with
<azonenberg>
something this obscure, when the entire production lot was made at my request and it was probably the first time this SKU was ever ordered?