Rendering and updating a colormap
I'm developing a library related to real-time image generation. To test it, I'm trying to make an app that can convert a
Color[,]
into an image and display it, updating at 60 fps (Color is a class with 3 float fields).
Ideally, I'd want my code to be like this:
where imageDisplay would probably be a PictureBox in a WPF app (with which I have no experience at all).
(This is of course pseudocode)
Any help is greatly appreciated.75 Replies
@GUI
There's multiple ways you could go about this:
- Use a
WriteableBitmap
- Use Win2D
and a canvas control/canvas image source, and draw manually
- Use ComputeSharp
- etc.
Also: I would really avoid allocating an array for each frame, that's uber expensiveIf it isn't too bothersome, could you please explain how exactly should I do it using a WriteableBitmap?
where imageDisplay would probably be a PictureBox in a Windows FormsYou've chosen a bad GUI framework and the worst possible control for this... If you want performance, at the very least, you should use .NET 6, rather than .NET Framework 4.x
As I've stated, I have no experience whatsoever with Windows Forms.
I am using .NET 6, I've never mentioned .NET Framework
Ok, but generally, the underlying GUI framework you want to use is going to define all the other capabilities available to you.
As much as I hate UWP, it still is more performant than anything else.
Personally, I would suggest using WPF, but this will depend on availability of features Sergio mentioned before
Technically speaking, if you know what you're doing, there are ways to make WinForms apps very fast, but most of the times that requires using low-level APIs.
I'm using WPF, which I thought wasthe same as Windows Forms, but it seems I'm mistaken
No, WPF is quite different and it's way more capable than WinForms.
But WPF has it's own limitations.
Something that wasn't yet mentioned here is SkiaSharp.
IIRC, that lib has some features that may interest you.
This app is only to test a library that I'm developing, so it doesn't need to be anything fancy, all I want to do is to 'display' a 2d array of colours that changes over time
Then you probably want to try using
ComputeShart
or WritableBitmap
Well, that comes back to what I asked, WriteableBitmap seems to be the best option for my use case, but I don't know how to use it and documentation isn't clear either.
I think @Klarth used it and may be able to help with that.
Then I'll wait for @Klarth to come online
Check out https://github.com/stevemonaco/WpfRealTimeBitmap
I also have it ported to Avalonia: https://github.com/stevemonaco/AvaloniaDemos/tree/master/RealTimeBitmapAdapter
It's very close to what you want and the approach should map pretty well to UWP, too, if you want to do that.
I could have skipped the indirection of creating a separate color map (with pixel values [0, 1]) and directly written randomized grayscale to the
WriteableBitmap
, but that didn't fit my scenario.It seems that my project can't access System.Windows.Media, it says 'missing assembly reference'. How can I add a reference to PresentationCore.dll?
This is a WPF app, not a WinForms app.
I strongly suggest you not use WinForms, especially if this is basically all the functionality you need.
Ah, that confusion has already been sorted out, my project is indeed a WPF app
There's no reason why you should have to add a reference to that if you've created a WPF .NET 6 app.
Are you sure you didn't accidentally use the WPF .NET Framework project template?
According to the properties window, I'm using .NET 6, not .NET Framework
I'm not sure how that could be the case. š¤
Can you paste your .csproj?
Wait.. What!!??
I clearly remember choosing WPF, and yet the properties window looks like this:
Is it enough to just swap them, or should I create a new project?
If you don't have XAML files (eg. App.xaml, etc) in your project, then recreate because you didn't create a WPF project.
Otherwise, it might be ok to swap. I'd probably just recreate though to save potential headaches.
I see, I'll recreate it then, no XAML on my end...
So... uh... how does WPF work?????
$rulesofwpf
You write XAML for layout / styling. You can use code-behind, too, but it should be limited in scope.
The designer sucks compared to WinForms. I disable it and suggest that newcomers disable too, after a few days.
So how can I replicate this kind of design in XAML? (it's an image and two text boxes)
Start with this š
A bit more in-depth... https://paste.mod.gg/cwmsfvtamagx/0
Thanks a lot for taking the time to do this! I'll try to understand what you're doing, it shouldn't be to hard.
The layout philosophy is a lot different from WinForms, if you were used to that.
Good thing that I wasn't then
WPF uses layout controls with their own sizing characteristics instead of the anchoring system that WinForms uses.
Basically, this is a 3x3 grid with proportional sizing.
You should be able to learn a lot by going through this site https://wpf-tutorial.com/
The code is actually quite straightforward, except for one part: What are the asterisks after some of the sizes in the Row/ColumnDefinitions?
Thanks, I'll check it out
If they sum to 100, like in this case, you can think of them as percentages of the remaining proportional space.
If you had two cells only, one with 2* and the other with *, then the sum is 3. So the first would be 66% and the second would be 33%.
So they are fractional relative units, I see. Nothing too out there, I've seen them before.
In this, you use a
Binding
for the image source. How do those work?That's a lot to go through.
Roughly, bindings associate a property on the
DataContext
with a dependency property on a control.
There are numerous configurations ranging from indirection and how/if the updates occur.What's the purpose of a BitmapAdapter?
Self-contained type to manage the copying from the native image concept
ColorScaleImage
to the WPF image concept WriteableBitmap
.
Well, that's the specific ColorScaleBitmapAdapter
. BitmapAdapter
is a general abstraction.
Keep in mind, these are patterns for a medium-sized app that is very generalizable. So it might not make sense to you when you're trying to scaffold a new, tiny project.
I could remove all of the abstractions and just do everything in one code-behind file, but that doesn't scale with project complexity.What is the
!.
operator in this line?$handlenullable
If you're getting warnings related to nullables, here's how you can handle it.
1. Forward
Just let it go š¶
2. Suppress
By adding !, like here:
3. Handle null
or
Part of null-forgiving operator.
In this case, shouldn't it be
Using the null-forgiving member-access operator?
Probably. It's a small demo that was updated from an older C#, so I don't care that much.
Fair enough
I'm not sure how I can convert my Image class to a WriteableBitmap... Could you please check the code at https://github.com/levi-gomes/RIRE and give me some clues?
GitHub
GitHub - levi-gomes/RIRE
Contribute to levi-gomes/RIRE development by creating an account on GitHub.
I've of course looked at
ColorScaleBitmapAdapter.Render
, but I don't quite understand what's going on...Before that, what is https://github.com/levi-gomes/RIRE/blob/master/RIRE/Color.cs ?
Is this just one pixel? Or is it many?
It's a colour - so I guess it is a single pixel.
Ok, so this is going to absolutely tank the performance. First, it should be a
struct
and not a class
. As-is, your image array is an array of pointers, so there's a lot of extra allocations + indirection.
I'm guessing each color channel is normalized between 0-1?Yes, it is
I'd start with something more like:
You can remove the
readonly
and make it mutable if you want...but that will solve most of the perf issues that will happen.There's no harm in keeping the indexer, is there?
There is...you don't want to store an array...because of even more allocation+indirection costs.
You could do a
fixed
array, but I'd recommend against it. It's not sensible to index a single pixel, IMO.I've refactored it as such:
Is this fine? Or should I completely get rid of that indexer?
I would get rid of it. You're going to have issues with value-type copy semantics at some point, I think.
struct
should typically be immutable because of that.Well, I don't think I'll really need it, so I'll take it out. As for making it
readonly
, would it still allow for something like some_color.r+=0.1f
?No...because it wouldn't be readonly if you could modify it.
Then I'll try to get away with it being mutable, if I do get issues I'll change to immutable
var newColor = new Color(some_color.R + 0.1f, some_color.G, some_color.B);
You could add some methods to clean it up.Alright, I'll make it
readonly
and deal with it. That can be a future me problem.https://paste.mod.gg/cxhounnbbfcs/0
So your implementations would look something like that.
It's more efficient to use
Span<T>
than your multidimensional array, but your image type doesn't support that. Some of the perf gap will be closed in .NET 7 with MD arrays.So I would add this to
MainWindow
and invoke it from RenderFrame
, am I right?How you integrate it depends on you.
That's what I did, but I didn't want to use MVVM in the WPF version.
Sorry, what is MVVM?
The Avalonia version is a bit more MVVM.
Model-View-ViewModel architecture. It's a way of separating concerns to make individual components more simplified at the expense of having many more components.
Unfortunately I have to go now. I'll try to integrate it at a later date (probably tomorrow). I'll let you know if I manage to do it (or if I have any more questions...). I can't thank you enough for your help!
Compute... Shart?
The docs show an example. It's a bit awkward as the API isn't the best, but it does the trick. Essentially you just have a grid of pixels, so you can lock the bitmap, update them, unlock the bitmap, every frame: https://learn.microsoft.com/en-us/dotnet/api/system.windows.media.imaging.writeablebitmap
WriteableBitmap Class (System.Windows.Media.Imaging)
Provides a BitmapSource that can be written to and updated.
Also what framework are you using?
I've got it working!