❔ Converting between various kinds of Bitmap objects without using Async code
Hi all,
I need help doing the following:
#1) Convert from an System.Drawing.Bitmap object to an Windows.Graphics.Imaging.SoftwareBitmap object without using asynchronous code (no "await" code, no "async" code, etc) and without needing to temporarily store the data on the filesystem as part of the process.
The image data always originates as an System.Drawing.Bitmap object that is created by taking a screenshot users' screen or a rectangular portion of a monitor.
I need to take that System.Drawing.Bitmap object and convert it to a Windows.Graphics.Imaging.SoftwareBitmap because I need to pass the image to a library that expects the image data to be a Windows.Graphics.Imaging.SoftwareBitmap. I do not have control over how that library works, I need to use it as-is.
#2) Convert from an Windows.Graphics.Imaging.SoftwareBitmap object to an System.Drawing.Bitmap object without using asynchronous code (no "await" code, no "async" code, etc) and without needing to temporarily store the data on the filesystem as part of the process.
I assume that I will need to be able to convert from an Windows.Graphics.Imaging.SoftwareBitmap object to an System.Drawing.Bitmap object because the System.Drawing.Bitmap object allows me to save it to the filesystem very easily while the Windows.Graphics.Imaging.SoftwareBitmap object does not allow saving to the filesystem easily. I want to store the Windows.Graphics.Imaging.SoftwareBitmap object on the filesystem as part of #3 below.
61 Replies
#3) Save an Windows.Graphics.Imaging.SoftwareBitmap object to the filesystem as a .BMP file, ideally with as few conversions to other different objects as possible. Also, I need the file to end up as a .BMP file on the filesystem, but I don't want to store it on the filesystem as any other kind of file temporarily as part of the conversion process prior to the final save as a .BMP file.
I need to be able to save the Windows.Graphics.Imaging.SoftwareBitmap to the filesystem so that I can verify that the conversion from the System.Drawing.Bitmap to the Windows.Graphics.Imaging.SoftwareBitmap (in #1 above) was successful.
#4) Notes:
I want to avoid using asynchronous code (no "await" code, no "async" code, etc) because using asynchronous code prevents me from debugging easily in Visual Studio 2022 because it stops me from being able to step through the code one line at a time.
I want to avoid any unnecessary conversions and and unnecessary storing to the filesystem for the sake of performance.
I have been searching online and have tried a bunch of different things with minimal success. I will post the code that I am working with as a response to my initial post.
I am hoping for working code snippets or an existing library that I can use, but any help is very appreciated, thanks!
For #1 I am currently using the following code:
public SoftwareBitmap SoftwareBitmapFromBitmap(Bitmap bitmap)
{
// TODO: Test
async Task<SoftwareBitmap> LocalAsyncFunc(Bitmap bitmap)
{
using (var stream = new Windows.Storage.Streams.InMemoryRandomAccessStream())
{
bitmap.Save(stream.AsStream(), ImageFormat.Bmp);
Windows.Graphics.Imaging.BitmapDecoder decoder = await Windows.Graphics.Imaging.BitmapDecoder.CreateAsync(stream);
SoftwareBitmap softwareBitmap = await decoder.GetSoftwareBitmapAsync();
return softwareBitmap;
}
}
var task = LocalAsyncFunc(bitmap);
task.Wait();
return task.Result;
}
Async code will only make debugging hard if you're doing something stupid like using
async void methodsFor #2 and #3 I am using the following code:
public void StoreSoftwareBitmapAsBitmapFileOnDisk(SoftwareBitmap softwareBitmap, string softwareBitmapScreenshotPath)
{
// TODO: Test
async Task LocalAsyncFunc(SoftwareBitmap softwareBitmap, string softwareBitmapScreenshotPath)
{
using (var stream = new Windows.Storage.Streams.InMemoryRandomAccessStream())
{
//Windows.Graphics.Imaging.BitmapEncoder encoder = await Windows.Graphics.Imaging.BitmapEncoder.CreateAsync(Windows.Graphics.Imaging.BitmapEncoder.PngEncoderId, stream);
Windows.Graphics.Imaging.BitmapEncoder encoder = await Windows.Graphics.Imaging.BitmapEncoder.CreateAsync(Windows.Graphics.Imaging.BitmapEncoder.BmpEncoderId, stream);
//Windows.Graphics.Imaging.BitmapEncoder encoder = await Windows.Graphics.Imaging.BitmapEncoder.CreateAsync(Windows.Graphics.Imaging.BitmapEncoder.BmpEncoderId, stream);
encoder.SetSoftwareBitmap(softwareBitmap);
try
{
await encoder.FlushAsync();
}
catch (Exception err)
{
Log($"StoreSoftwareBitmapAsBitmapFileOnDisk() :: await encoder.FlushAsync() :: err.HResult: {err.HResult}");
}
finally
{
Log($"StoreSoftwareBitmapAsBitmapFileOnDisk() :: await encoder.FlushAsync() :: No Exception Thrown");
}
using (Bitmap bitmap = new System.Drawing.Bitmap(stream.AsStream()))
{
bitmap.Save(softwareBitmapScreenshotPath);
}
}
}
var task = LocalAsyncFunc(softwareBitmap, softwareBitmapScreenshotPath);
task.Wait();
}
Unfortunately all of my code currently is asynchronous which I want to avoid - I don't want it to be asynchronous.
BitmapEncoder has no synchronous methods
So you can either do it all properly, like it should be done, with async and await
Or you write garbage-tier code with blocking, .GetAwaiter().GetResult() and shitHi ZZZZZ, I am not using async void methods. That said, if I try to step through my code in the debugger one line at a time using F10 Visual Studio will not let me once I try to use F10 on a line containing asynchronous code. That is my issue with asynchronous code. This video https://www.youtube.com/watch?v=7POKQgdkrA4 from Microsoft shows the issue I am running into.
Microsoft Visual Studio
YouTube
Why is Async code hard to debug?
Isadora Rodopoulos, an engineer on the Visual Studio Debugging team, joins us to kick off a mini-series on debugging async code. Today she explains how async code works behind the scenes and provides some insights into why it is hard to debug.
Next week, Isadora will explore tooling in Visual Studio to debug async code.
Resources:
Demo Source...
If there is a way to do it asynchronously without having the issues I described I am happy to do it, but I don't think you can. The way MS suggests to deal with async code in the video I linked is terrible.
Well, then
.GetAwaiter().GetResult() is as lesast-bad as you can haveI don't want to write the code using .GetAwaiter().GetResult() and the like. I want to write the code by manipulating the underlying byte arrays.
If a library only has async methods, it only has async methods
In this case,
BitmapEncoder only has async methodsYes, I expect to not be able to only use Microsoft's libraries, I expect to need a third part library (though I have not found one yet) OR write a small library myself that manipulates the underlying byte arrays.
ImageSharp perhaps
Ok, I will take a look at the ImageSharp library
It looks like ImageSharp unfortunately does not support SoftwareBitmaps as per https://github.com/SixLabors/ImageSharp/issues/435
GitHub
Library Inter-op extensions · Issue #435 · SixLabors/ImageSharp
ImageSharp should provide an initial standard structure for adding library inter-op apis. ImageSharp <=> System.Drawing interop ** Simple API the allows for converting System.Drawing....
For these image interop tasks, you typically take the source image (
Bitmap), create an equally sized destination image (SoftwareBitmap) with the same pixel color format, then copy the source pixel buffer into the destination pixel buffer.
https://learn.microsoft.com/en-us/uwp/api/windows.ui.xaml.media.imaging.writeablebitmap.pixelbuffer?view=winrt-22621 possibly shows one half, but you might need to use loops rather than a buffer copy, depending.
I guess it's https://learn.microsoft.com/en-us/uwp/api/windows.graphics.imaging.softwarebitmap.copyfrombuffer?view=winrt-22621. The docs misled me into WriteableBitmap, though it may be workable depending on what you're actually trying to do.Hi Klarth, hmmmm.... let me give that a try
I would probably use
Bitmap.LockBits to get access to a pixel buffer (you'll need unsafe code because this is a pointer, not an array), create a SoftwareBitmap with the right size / pixel format, then use a loop to copy into the buffer you get from SoftwareBitmap.LockBuffer. You might be able to skip the loop by using Marshal.Copy instead.I am trying to implement this option that you mentioned first, then I will look at the LockBits related option you mentioned 🙂
IIRC, you can't efficiently get access to a
Bitmap's pixel buffer without LockBits and it's not an IBuffer anyways. So I think that rules out CopyFromBuffer.Well, I think I need to go Bitmap -> SourceBitmap -> WriteableBitmap (which supports PixelBuffer), then I think I can use CopyFromBuffer to create the SoftwareBitmap. No?
No, do this.
It's
Bitmap -> SoftwareBitmap once you have the pixel buffers from each.hrmmm, I tried to do something like that earlier, but I wasn't sure how to correctly convert from a System.Drawing.Imaging.PixelFormat (which Bitmap uses) and a Windows.Graphics.Imaging.BitmapPixelFormat (which SoftwareBitmap uses). I looked earlier and found similar conversion code online, but not for those two. The similar code I found was https://github.com/mathieumack/PixelFormatsConverter/blob/master/PixelFormatsConverter/PixelFormatsConverter/PixelFormatConverterExtensions.cs but those are for slightly different types.
GitHub
PixelFormatsConverter/PixelFormatConverterExtensions.cs at master ·...
Allow a convertion from enum System.Drawing.Imaging.PixelFormat to System.Windows.Media.PixelFormats and inversion. - PixelFormatsConverter/PixelFormatConverterExtensions.cs at master · mathieumack...
You're going to have a bad time if you need to convert between all pixel formats.
You should work with an input
Bitmap with one known pixel format if you can force that.Ok, let me take a look at what pixel format my Bitmap is using
It's likely 32bit bgra.
The PixelFormat for my bitmap is Format32bppArgb
https://learn.microsoft.com/en-us/dotnet/api/system.drawing.imaging.pixelformat?view=windowsdesktop-7.0 shows the enum for bitmap.PixelFormat
PixelFormat Enum (System.Drawing.Imaging)
Specifies the format of the color data for each pixel in the image.
https://learn.microsoft.com/en-us/uwp/api/windows.graphics.imaging.bitmappixelformat?view=winrt-22621 show the enum for Windows.Graphics.Imaging.BitmapPixelFormat for the SoftwareBitamp
BitmapPixelFormat Enum (Windows.Graphics.Imaging) - Windows UWP app...
Specifies the pixel format of pixel data. Each enumeration value defines a channel ordering, bit depth, and data type.
but how the heck do I know which one maps up to the other. I don't see one that would seem to correspond to Format32bppArgb
BitmapFormat.Bgra8 probably.
Else it's Rgba8
https://learn.microsoft.com/en-us/uwp/api/windows.graphics.imaging.bitmappixelformat?view=winrt-22621But aren't those for pixels where each color is only 8 bits and it seems like the ones for my Bitmap are 32 bits for each color, no?
Either way, it doesn't matter. Use
Bgra8 and you can do the color channel shifting in code if you need to. That will necessitate a loop rather than Marshal.Copy, but it's how most of my conversions are done.
No, the naming is just inconsistent.
They're both 32bits per pixel, 8 bits per color channel.You are right, they are both 32bits per pixel and 8 bits per color channel. Does it make sense to try to get the Bitmap to use a different pixelformat that matches my pixelformat options for the SoftwareBitmap so that I can try to still use Marshal.Copy?
Ideally, they would match. It's not really a big deal unless it's video and you're targeting lower end machines.
I would spend some time doing the actual conversion first though. Worry about optimization later, if necessary.
Ok, I am working on this "I would probably use Bitmap.LockBits to get access to a pixel buffer (you'll need unsafe code because this is a pointer, not an array), create a SoftwareBitmap with the right size / pixel format, then use a loop to copy into the buffer you get from SoftwareBitmap.LockBuffer. You might be able to skip the loop by using Marshal.Copy instead." now
public enum BitmapAlphaMode
{
//
// Summary:
// The alpha value has been premultiplied. Each color is first scaled by the alpha
// value.
Premultiplied,
//
// Summary:
// The alpha value has not been premultiplied. The alpha channel indicates the transparency
// of the color.
Straight,
//
// Summary:
// The alpha value is ignored.
Ignore
}
Which of those BitmapAlphaMode's should I be using for my SoftwareBitmap?
Straight, probably.
Either way, you can change it later. It will only make the colors a bit off if you get it wrong.
Ok
that's not entirely true
async/await makes debugging more complicated sometimes
public unsafe SoftwareBitmap SoftwareBitmapFromBitmap3(Bitmap bitmap)
{
// "I would probably use Bitmap.LockBits to get access to a pixel
// softwareBitmapBitmapBuffer (you'll need unsafe code because this is a pointer, not an
// array), create a SoftwareBitmap with the right size / pixel format, then use a loop
// to copy into the softwareBitmapBitmapBuffer you get from SoftwareBitmap.LockBuffer.
// You might be able to skip the loop by using Marshal.Copy instead, but only if the
// PixelFormats match."
//The PixelFormat for my bitmap is Format32bppArgb
SoftwareBitmap softwareBitmap = new SoftwareBitmap(BitmapPixelFormat.Bgra8, bitmap.Width, bitmap.Height, BitmapAlphaMode.Straight);
using (BitmapBuffer softwareBitmapBitmapBuffer = softwareBitmap.LockBuffer(BitmapBufferAccessMode.Write))
{
using (var reference = softwareBitmapBitmapBuffer.CreateReference())
{
byte* dataInBytes;
uint capacity;
((IMemoryBufferByteAccess)reference).GetBuffer(out dataInBytes, out capacity);
BitmapPlaneDescription bufferLayout = softwareBitmapBitmapBuffer.GetPlaneDescription(0);
for (int i = 0; i < bufferLayout.Height; i++)
{
for (int j = 0; j < bufferLayout.Width; j++)
{
dataInBytes[bufferLayout.StartIndex + bufferLayout.Stride * i + 4 * j + 0] = 0; //B
dataInBytes[bufferLayout.StartIndex + bufferLayout.Stride * i + 4 * j + 1] = 0; //G
dataInBytes[bufferLayout.StartIndex + bufferLayout.Stride * i + 4 * j + 2] = 0; //R
dataInBytes[bufferLayout.StartIndex + bufferLayout.Stride * i + 4 * j + 3] = 0; //A
}
}
}
}
return softwareBitmap;
}
// Using the COM interface IMemoryBufferByteAccess allows us to access the underlying byte
// array in an AudioFrame
[ComImport]
[Guid("5B0D3235-4DBA-4D44-865E-8F1D0E4FD04D")]
[InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
private unsafe interface IMemoryBufferByteAccess
{
void GetBuffer(out byte* buffer, out uint capacity);
}
Does that seem reasonable for the part that iterates over the SoftwareBitmap so far?
sadly, I agree 100%
Looks alright. I prefer to move the pointer rather than recalculate that much, but you have a working approach assuming no mistakes I've overlooked.
Now you need to match it up with the data from the
Bitmap.Excellent, working on that now
Ok, the message.txt above is my updated code, what do you think? Anything wrong / missing?
I don't bother with the incredibly long names. Sometimes I don't deal with individual color channels either if you can help it. ie. Cast to
uint* and work in that instead.
You do have some extra complications, but if you work them out...you can do something relatively straightforward like https://github.com/stevemonaco/ImageMagitek/blob/main/TileShop.Avalonia/ViewExtenders/Imaging/DirectBitmapAdapter.cs#L78
Ah, this older code is probably a bit more straightforward. I don't really have any custom domain logic inside: https://github.com/stevemonaco/ImageMagitek/blob/wpf/TileShop.WPF/ViewExtenders/Imaging/ImageRgba32Source.cs#L43
You don't have Span<T> with any of the types you're working with, but it's close enough to demonstrate.Cool, taking a look at the links you sent me
hrmmm, I am slowly trying to adapt parts of the way you are doing it for my code. In testing (before I started adapting my code) I noticed that bitmapData.Scan0 causes an overflow on the int that I assign it to. I changed the variable to a uint and it still causes an overflow. Is bitmapData.Scan0 not the pointer to the beginning of the pixel data in the Bitmap or something?
It is the beginning address of the pixel data.
Then why would that value be larger than a uint?
I'd suspect moreso it's out-of-bounds access.
The specific error is "An exception of type 'System.OverflowException' occurred in System.Private.CoreLib.dll but was not handled in user code
Arithmetic operation resulted in an overflow."
And even the simplified line "AlphaOffsetInBitmapDataInBytes = (uint)(bitmapData.Scan0); //A" still causes the same error
Do I need to be running Visual Studio as Admin or something?
Oh, you can't do that.
That's a pointer, not a value.
You need to dereference it to get the value with the
* operator.Ok, giving it a try
When I write the simplified line (does not account for row nor column) like this:
AlphaOffsetInBitmapDataInBytes = (uint)((byte*)bitmapData.Scan0); //A
it no longer overflows. Does that look correct to you?
I don't think there's a reason to cast it to
uint. You only use uint if you're dealing with the entire color. Stick with byte if you're separating the color into color channels.
But yes, that's how you would get the very first byte of the pixel buffer. (You'll need to adjust the pointer offset later)hrmmm, but shouldn't AlphaOffsetInBitmapDataInBytes be a uint since I am using AlphaOffsetInBitmapDataInBytes as an index into dataInBytes?
I think maybe you are saying I am doing a needless cast and that I should grab the byte that represents a given color channel without making a separate value that contains the index of that byte
Yes, just do:
byte AlphaOffsetInBitmapDataInBytes = *bitmapData.Scan0); //A
Might be *(bitmapData.Scan0). Not sure how precedence works there.
But really, I store bitmapData.Scan0 into a different pointer variable so I can move it. It's cumbersomely long.Do you really mean it should be byte AlphaColorValueInBitmap = *bitmapData.Scan0); //A
i.e. what you are proposing would get me the COLOR VALUE directly, instead of the INDEX?
Why would you want the index?
well, I have "byte* dataInBytes" and the value I was trying to get would be an index into those bytes, no?
or maybe I am thinking of it as a byte array when really it is a pointer to a byte
If you're calculating an index, you don't need a pointer.
The index is just (x + x * y) * 4.
Right. I think I was making the mistake of thinking that I was dealing with a byte array where I would want to know the index but in reality I am dealing with a pointer and there is no index to calculate
To the start of the color anyways. Assuming we aren't considering bitmap layout...which are stored in a nonobvious ordering.
hrmmm, Ok, I am thinking about what you said
Klarth, thank you for all of your help, I am a lot further along now than I was before talking to you. I think I need a break, but I will think more about what you said and tackle this again later. Thanks again
Was this issue resolved? If so, run
/close - otherwise I will mark this as stale and this post will be archived until there is new activity.