C
C#2y ago
exergist

✅ Quicker Screenshot

I'm trying to quickly capture a screenshot. My current method (when capturing the entirety of a 3440x1440p screen) takes about 60-80ms:
// Capture screenshot
static System.Drawing.Bitmap GetScreenshot(System.Drawing.Rectangle bounds = default)
{
if (bounds == default)
bounds = Screen.GetBounds(System.Drawing.Point.Empty);

System.Drawing.Bitmap bitmap = new System.Drawing.Bitmap(bounds.Width, bounds.Height);
using (System.Drawing.Graphics g = System.Drawing.Graphics.FromImage(bitmap))
{
g.CopyFromScreen(System.Drawing.Point.Empty, System.Drawing.Point.Empty, bounds.Size);
}
return bitmap;
}
// Capture screenshot
static System.Drawing.Bitmap GetScreenshot(System.Drawing.Rectangle bounds = default)
{
if (bounds == default)
bounds = Screen.GetBounds(System.Drawing.Point.Empty);

System.Drawing.Bitmap bitmap = new System.Drawing.Bitmap(bounds.Width, bounds.Height);
using (System.Drawing.Graphics g = System.Drawing.Graphics.FromImage(bitmap))
{
g.CopyFromScreen(System.Drawing.Point.Empty, System.Drawing.Point.Empty, bounds.Size);
}
return bitmap;
}
Capturing a smaller region of the screen (via bounds) definitely helps. Thanks!!
135 Replies
Buddy
Buddy2y ago
What are you making?
FusedQyou
FusedQyou2y ago
60-80ms for a 3440x1440p screen does not sound that bad? Maybe limit it to a certain resolution instead of depending on what is possible
Jester
Jester2y ago
GDI is horribly slow. try the Desktop Duplication API its vsynced (so its as fast as it has to be) thr only problem is you will need to pinvoke it into c# yourself or find a library that already did that
Jester
Jester2y ago
Desktop Duplication API - Win32 apps
Windows 8 disables standard Windows 2000 Display Driver Model (XDDM) mirror drivers and offers the desktop duplication API instead.
exergist
exergist2y ago
A personal project for image detection (doesn't run continuously) that I'm trying to better optimize Any chance you know of a good C# wrapper for it? I'm still a noob with C# (and way more so C++) and this is a great learning opportunity.
sibber
sibber2y ago
directx would probably be the fastest
exergist
exergist2y ago
super high speed isn't absolutely necessary (this isn't a live screen capture application) just trying to keep things (relatively) lightweight but trying to see what the next level of "better" is
Jester
Jester2y ago
yeah thats the desktop duplication api for
Jester
Jester2y ago
@Exergist i think ive found something you could use. https://github.com/nnn149/DesktopDuplicationWapper
GitHub
GitHub - nnn149/DesktopDuplicationWapper: C# wrapper for the Deskto...
C# wrapper for the Desktop Duplication Api. Contribute to nnn149/DesktopDuplicationWapper development by creating an account on GitHub.
Jester
Jester2y ago
its a wrapper on top of a wrapper on top of a wrapper of a wrapper
sibber
sibber2y ago
oh its looks like its a wrapper that uses directx
Groophy
Groophy2y ago
I saw firstly 'reference' in my more than four years programming experience
exergist
exergist2y ago
Unfortunately one of the DesktopDuplicationWrapper dependencies (Vortice.Direct3D11) has an issue when installing via NuGet Unable to find a version of 'SharpGen.Runtime' that is compatible with: 'Vortice.Direct3D11 2.2.0 constraint: SharpGen.Runtime (>= 2.0.0-beta.11)', 'Vortice.DirectX 2.2.0 constraint: SharpGen.Runtime (>= 2.0.0-beta.11)', 'Vortice.DXGI 2.2.0 constraint: SharpGen.Runtime (>= 2.0.0-beta.11)' I submitted an issue for the repo, but I'm not sure what to do from here 🤔 It looks like the available NuGet packages for SharpGen.Runtime and SharpGen are at v1.2.1 and v1.2.0 respectively. Any recommendations?
Jester
Jester2y ago
get the preview? thats a checkbox somewhere in thr nuget download screen
exergist
exergist2y ago
OK I'll check for it tonight
Accord
Accord2y ago
Was this issue resolved? If so, run /close - otherwise I will mark this as stale and this post will be archived until there is new activity.
exergist
exergist2y ago
After following the suggestion by R'tsej and doing some more fiddling with pre-installing required dependencies via NuGet I finally got the DesktopDuplicationWapper components installed. Unfortunately upon executing the sample code the captured content is just a black image, so I'm going to submit an issue to that repo.
exergist
exergist2y ago
In the mean time I'll probably give this version a try https://github.com/nicolasdeory/desktop-duplication-net
GitHub
GitHub - nicolasdeory/desktop-duplication-net: Capture the desktop ...
Capture the desktop in real time with .NET Core using the Windows 8 Desktop Duplication API - GitHub - nicolasdeory/desktop-duplication-net: Capture the desktop in real time with .NET Core using th...
Jester
Jester2y ago
i might have some code lying around, ill take a look for you
exergist
exergist2y ago
unfortunately I get the same behavior (black image) with the desktop-duplication-net repo (though I also realized that I need to target .NET Framework, so I can't use it anyway)
Jester
Jester2y ago
black image? thats weird <:PES_Think:639363477458255874> can i see your code?
exergist
exergist2y ago
public static System.Drawing.Bitmap DuplicateDesktopScreenShot()
{
System.Drawing.Bitmap bitmap;
using (DesktopDuplicator desktopDuplicator = new DesktopDuplicator(0, 0))
{
Thread.Sleep(10); // Seems to make image acquisition more consistent (non-black image)
DesktopFrame frame = desktopDuplicator.GetLatestFrame(0, 0, 1920, 1080);
if (frame != null)
{
bitmap = frame.DesktopImage;
bitmap.Save("helloDesktop.png");
return bitmap;
}
else
return null;
}
}
public static System.Drawing.Bitmap DuplicateDesktopScreenShot()
{
System.Drawing.Bitmap bitmap;
using (DesktopDuplicator desktopDuplicator = new DesktopDuplicator(0, 0))
{
Thread.Sleep(10); // Seems to make image acquisition more consistent (non-black image)
DesktopFrame frame = desktopDuplicator.GetLatestFrame(0, 0, 1920, 1080);
if (frame != null)
{
bitmap = frame.DesktopImage;
bitmap.Save("helloDesktop.png");
return bitmap;
}
else
return null;
}
}
@Jester Without that small pause I get a black image 90% of the time. To be clear, I'm back to using DesktopDuplicationWapper.
sibber
sibber2y ago
that means you have a sync issue
Jester
Jester2y ago
@Exergist im pretty sure the second frame you get will always be fine. its the very first frame thats often black in this api, ive noticed
sibber
sibber2y ago
thread.sleep is never the solution
Jester
Jester2y ago
yeah remove the sleep only the first frame will be black, all the others will work fine in my code using it i dont use the very first frame either, but i do use every one after that i think the api always syncs automatically
sibber
sibber2y ago
oh idk why then
exergist
exergist2y ago
public static System.Drawing.Bitmap DuplicateDesktopScreenShot()
{
using (DesktopDuplicator desktopDuplicator = new DesktopDuplicator(0, 0))
{
DesktopFrame frame1 = desktopDuplicator.GetLatestFrame(0, 0, 1920, 1080);
DesktopFrame frame2 = desktopDuplicator.GetLatestFrame(0, 0, 1920, 1080);
if (frame1 != null)
frame2.DesktopImage.Save("helloFrame1.png");
if (frame2 != null)
{
System.Drawing.Bitmap bitmap = frame2.DesktopImage;
bitmap.Save("helloFrame2.png");
return bitmap;
}
else
return null;
}
}
public static System.Drawing.Bitmap DuplicateDesktopScreenShot()
{
using (DesktopDuplicator desktopDuplicator = new DesktopDuplicator(0, 0))
{
DesktopFrame frame1 = desktopDuplicator.GetLatestFrame(0, 0, 1920, 1080);
DesktopFrame frame2 = desktopDuplicator.GetLatestFrame(0, 0, 1920, 1080);
if (frame1 != null)
frame2.DesktopImage.Save("helloFrame1.png");
if (frame2 != null)
{
System.Drawing.Bitmap bitmap = frame2.DesktopImage;
bitmap.Save("helloFrame2.png");
return bitmap;
}
else
return null;
}
}
This seems to work more reliably. Though at the time of testing frame1 is delivering an image as expected 🤔
Jester
Jester2y ago
i think bcuz it gets the frames from the DWM and the DWM doesnt always keep an image ready for this api until someone uses it? <:PES_Think:639363477458255874>
exergist
exergist2y ago
So with the following:
public static System.Drawing.Bitmap DuplicateDesktopScreenShot()
{
using (DesktopDuplicator desktopDuplicator = new DesktopDuplicator(0, 0))
{
DesktopFrame frame = desktopDuplicator.GetLatestFrame(0, 0, 1920, 1080); // Initialize, but ignore this first frame
frame = desktopDuplicator.GetLatestFrame(0, 0, 1920, 1080); // Use the second frame going forward
if (frame != null)
{
/// frame.DesktopImage.Save("helloFrame.png"); // debug
return frame.DesktopImage;
}
else
return null;
}
}
public static System.Drawing.Bitmap DuplicateDesktopScreenShot()
{
using (DesktopDuplicator desktopDuplicator = new DesktopDuplicator(0, 0))
{
DesktopFrame frame = desktopDuplicator.GetLatestFrame(0, 0, 1920, 1080); // Initialize, but ignore this first frame
frame = desktopDuplicator.GetLatestFrame(0, 0, 1920, 1080); // Use the second frame going forward
if (frame != null)
{
/// frame.DesktopImage.Save("helloFrame.png"); // debug
return frame.DesktopImage;
}
else
return null;
}
}
The "screenshot time" for a 1080p desktop is averaging about 250-300 ms, which is slower than the method I originally posted. I'm guessing that's not expected?
Jester
Jester2y ago
with or without saving it? it would be weird if getframe would take more than 16ms on a 60fps display
exergist
exergist2y ago
above code encapsulates the timing range (no image save) that's why i figured something was amiss
Jester
Jester2y ago
rewrite it in C++ awesome you create an adapter, then an output and device and devicecontext with it
sibber
sibber2y ago
GitHub
ScreenGrab · microsoft/DirectXTK Wiki
The DirectX Tool Kit (aka DirectXTK) is a collection of helper classes for writing DirectX 11.x code in C++ - ScreenGrab · microsoft/DirectXTK Wiki
Jester
Jester2y ago
with that output an output duplocator and then you make a few textures2ds blablabla i could change some of my code to make a sketchy c# wrapper for it
exergist
exergist2y ago
O_o lol that's probably out of scope for now I'd always be open to checking it out, but I don't want you to sink time into it unless you're keen on it. I figured there might be a low friction method to reducing screen capture time below ~60-80ms, but it seems like more investment would be needed. TLDR - if you really want to make a sketchy wrapper I'd certainly check it out, otherwise what I have right now is probably sufficient.
Jester
Jester2y ago
@Exergist here is the wrapper i made. it works perfectly on my machine. i get a frame every 1-16ms. and it doesnt need any nuget packages. (also the first frame isnt black and there are probably way less allocations than other wrappers out there) https://github.com/QubitTooLate/Snippies/blob/main/DesktopDuplicator.cs
GitHub
Snippies/DesktopDuplicator.cs at main · QubitTooLate/Snippies
A bunch of snippets, tips and tricks I learnt during my projects. - Snippies/DesktopDuplicator.cs at main · QubitTooLate/Snippies
exergist
exergist2y ago
Awesome!!! Ah but I'm limited to .Net Framework (C# 7.3)
Jester
Jester2y ago
Despair
exergist
exergist2y ago
Medium
Enabling and using C# 9 features on older and “unsupported” runtime...
A guide on how to enable support for many new C# 9 features on older runtimes and frameworks that do not offer them out of the box
exergist
exergist2y ago
Buddy
Buddy2y ago
Wait, Hmmm
exergist
exergist2y ago
i'm developing this as a plugin to run within the process of another app, which requires .Net Framework for interface
exergist
exergist2y ago
Okay this resolved a bunch of errors 🙂
exergist
exergist2y ago
<LangVersion>9.0</LangVersion>
exergist
exergist2y ago
@Jester all that remains is this error I must be missing something
Jester
Jester2y ago
you just need a way to have some unmanaged or memory there are plenty other ways to do it let me look
Jester
Jester2y ago
@Exergist try using this instead how is it going?
exergist
exergist2y ago
sorry had to step away for a second. i'm poking through and trying to swap for Marshal yep I'm struggling with this one. apologies, I'm not familiar with directly handling memory 😕
Jester
Jester2y ago
whats the issue? replace alloc frameBuffer = Marshal.AllocHGlobal(description.Width * 4 * description.Height).ToPointer(); replace free Marshal.FreeHGlobal(new IntPtr(frameBuffer));
exergist
exergist2y ago
getting close! no ToPointer for uint
Jester
Jester2y ago
uint?
exergist
exergist2y ago
Jester
Jester2y ago
a ( too many
exergist
exergist2y ago
Jester
Jester2y ago
hmmm on
exergist
exergist2y ago
brb, catching a train. appreciate the incredible help here
Jester
Jester2y ago
you should be able to figure it out yourself but here i already gave the answer wrong chat
exergist
exergist2y ago
ah yes you did! looks like this does the job frameBuffer = (void*)Marshal.AllocHGlobal(new IntPtr(description.Width * 4 * description.Height));
Jester
Jester2y ago
Hmm
exergist
exergist2y ago
🤔
Jester
Jester2y ago
uh oh
exergist
exergist2y ago
fails this assertion
Jester
Jester2y ago
looks like you dont have some part of directx installed or windows sdk or something idk
exergist
exergist2y ago
bwa?
Jester
Jester2y ago
idk but it might work without the debug flag
exergist
exergist2y ago
looks like the image gets disposed?
exergist
exergist2y ago
exergist
exergist2y ago
bitmap contains something that is 1080p, but test doesn't receive anything (and no image gets saved) wait nvm Looks like I cannot apply using to the bitmap But removing it works! I'll do some more testing tonight. Thank you everyone for your continued support! @Jester awesome snippet
Jester
Jester2y ago
thanks <:PES_Happy:493353112493621258> yeah returning a disposed bitmap isnt very useful ofc
exergist
exergist2y ago
Any suggestions for how i can monitor for memory leaks?
Jester
Jester2y ago
i honestly dont now. just write code that doesnt leak you know awesome
exergist
exergist2y ago
Lol well yea. I meant within the confines of this excellent snippet
exergist
exergist2y ago
CodeProject
Best Practices No. 5: Detecting .NET application memory leaks
In this article we are going to detect .NET application memory leaks.
exergist
exergist2y ago
@Jester yesterday I was doing my testing of your snippet via a trio of 1080p monitors (with one of them being the main target monitor for duplication). Today I tried it via three 1080p monitors and one 3440x1440 monitor (the 3440 is the main target monitor). Mentioning all this because today I encountered this error within the WriteFrameIntoBuffer method.
Jester
Jester2y ago
<:PES_Think:639363477458255874>
exergist
exergist2y ago
when i target the 1080p monitors the issue disappears
Jester
Jester2y ago
id like to see the values of width, height and rowpitch
exergist
exergist2y ago
standby
exergist
exergist2y ago
3440p target monitor
exergist
exergist2y ago
1080p target monitor
exergist
exergist2y ago
updated to include buffer in reality my current setup has: - one 3440x1440p - one 3840x2160p - two 1920x1080p I just happen to run the 4K monitor at 1080p while working. I tested the 4K monitor while running at 2560x1440p and that works fine. I then tried my 4K monitor running at 1680x1050 (16:10 ratio) and that does NOT work. I know very little about this area, but perhaps there's an issue with non-16:9 display ratios?
Jester
Jester2y ago
it could be maybe my pitch assumption is wrong yeah my code must have a few mistakes i wrote it half asleep do i even use my framebuffer field? <:PES_HuhWtf:642742545700618303> i do not so that can even be removed ill see if i can improve it in a few hours
Jester
Jester2y ago
Game Development Stack Exchange
Determine the stride of a DirectX Texture2D line?
Is there a way to determine, or preferably calculate/predict, the the stride of a line of a DirectX 11 Texture2D resource when using SharpDX? (E.g. Can we say the stride of a line is always a powe...
Jester
Jester2y ago
hmm this sucks but maybe if i do this once in the constructor i can assume it will be the same for every following frame
exergist
exergist2y ago
seriously @Jester don't lose sleep over this. the GDI method is FINE
Jester
Jester2y ago
thus removing the need for allocations i want to use my snippet myself too
exergist
exergist2y ago
ok then i'm onboard to help test along the way 😉
Jester
Jester2y ago
i would like a fast remote desktop for at home but its not as good as the nvidia one
exergist
exergist2y ago
also (at least for me) when i compare total bitmap capture time for DesktopDuplicator vs GDI the GDI is slightly faster (maybe by 20-40ms) granted i'm no pro at properly setting up "fair" performance tests
Jester
Jester2y ago
im not sure if he buffer that gets written to should be pinned or not
exergist
exergist2y ago
i do not intend to spam this, but i'm just looking for leaks
Jester
Jester2y ago
Despair i can believe the first frame being slower
exergist
exergist2y ago
again, don't apply a ton of weight to my feedback on my end the VS process memory definitely accumulates when running DesktopDuplicator 30 times nope it accumulates on my GDI version....
Jester
Jester2y ago
do you free the framebuffer in the dispose?
exergist
exergist2y ago
for DesktopDuplication yes?
Jester
Jester2y ago
yes
exergist
exergist2y ago
I'll say yes
Jester
Jester2y ago
lgtm dont allocate a new instance of this class for every screenshot. but i believe you really only needed one?
exergist
exergist2y ago
again my earlier comment was about MY GDI screenshot method memory seems fine for the DesktopDuplication
Jester
Jester2y ago
the gdi graphics.copyfromscreen ? or something
exergist
exergist2y ago
si
Jester
Jester2y ago
oh the gdi method leaks? not mine?
exergist
exergist2y ago
seemingly yes yours seems fine
Jester
Jester2y ago
my class didnt work for you at first, how did you fix that error?
exergist
exergist2y ago
which error specifically? DesktopDuplication still doesn't work for non-16:9 displays. I've just been continuing work by targeting my 1080p display.
Jester
Jester2y ago
this one
exergist
exergist2y ago
Hmm yes, got rid of the debug bits
Jester
Jester2y ago
the memory pitch is different for other ratio displays for gpu memory optimizations and i didnt think about that for example in hlsl shaders everything has to be like a multiple of 16 bits or bytes i think i do know the gdi api by heart as well getdc createcompatibledc createcompatiblebitmap selectobject bitblt getdibits releasedc deletedc deleteobj @Exergist i have pushed a fixed version of the code if you init the class when the application starts then you dont have to wait so long in the screenshot method itself
exergist
exergist2y ago
sweet i'll give that a try now 🙂 hmm i encountered a memory access issue that persists between runs. i'm going to restart and try again.
exergist
exergist2y ago
@Jester ok so this seems to work fine for 1080p (though I did get an example where the image was all black). Also works fine for 1440p screen. However when I target my 3440x1440p screen i CAN obtain the bitmap and save it, however upon further processing of the image (which again, works fine at 1080p and 1440p) I receive an error.
Jester
Jester2y ago
<a:aPES_CryDrink:678210605781352448> idk whats wrong
exergist
exergist2y ago
It's most interesting that the issue is limited to a particular screen (or size?) I'm afk now but IIRC I can switch target screen around and there's no issue. Later I can also try different resolution types via my 4k monitor if that would be helpful.
Jester
Jester2y ago
but the issue is outside of my snippet now? all my code works fine? except for the possible black screen?
exergist
exergist2y ago
Everything works fine with both versions except for 3440x1440p (though IIRC the errors were different between versions) And occasionally I get a black screen when it does work
Jester
Jester2y ago
where do you get the access error?
exergist
exergist2y ago
Doing some opencvsharp Mat processing
Jester
Jester2y ago
oh ok then my code works fine
exergist
exergist2y ago
It can handle a saved 3440p bitmap fine, but not one from the screenshot repo you shared
Jester
Jester2y ago
<:PES_Think:639363477458255874> the bitmap object? or just the bytes? weird if the bitmap object does work in one case but not the other can i see the code of you passing the bitmap to the opencv?
exergist
exergist2y ago
I can share later tonight
exergist
exergist2y ago
First I should point out I removed the using so that bitmap isn't disposed before I can actually use it
exergist
exergist2y ago
Second I had to once again remove the debug parts to get rid of a runtime error
exergist
exergist2y ago
private static Mat GetTargetMatFromScreen(int targetScreen, int targetAdapter = 0, Rectangle cropRegion = default)
{
Mat mat = null; // Initialize mat
using (Bitmap screenshot = GetDuplicationScreenshot(0, 0)) // Obtain screenshot of target display (via DesktopDuplication)
{
/// screenshot.Save("test.png"); // debug
if (cropRegion != default) // Check if a cropRegion was provided
{
using (Bitmap croppedScreenshot = CropImage(screenshot, cropRegion)) // Crop screenshot based on cropRegion and store in croppedScreenshot
{
if (croppedScreenshot != null) // Check if croppedScreenshot is NOT null
mat = BitmapConverter.ToMat(croppedScreenshot); // Create Mat converted from croppedScreenshot
}
}
else
{
mat = BitmapConverter.ToMat(screenshot); // Create Mat converted from screenshot Bitmap
if (mat == null) // Check if mat is null
Console.WriteLine("Could not create haystack Mat from screenshot. Image search canceled."); // Output info to log
}
}
return mat; // Return mat
}
private static Mat GetTargetMatFromScreen(int targetScreen, int targetAdapter = 0, Rectangle cropRegion = default)
{
Mat mat = null; // Initialize mat
using (Bitmap screenshot = GetDuplicationScreenshot(0, 0)) // Obtain screenshot of target display (via DesktopDuplication)
{
/// screenshot.Save("test.png"); // debug
if (cropRegion != default) // Check if a cropRegion was provided
{
using (Bitmap croppedScreenshot = CropImage(screenshot, cropRegion)) // Crop screenshot based on cropRegion and store in croppedScreenshot
{
if (croppedScreenshot != null) // Check if croppedScreenshot is NOT null
mat = BitmapConverter.ToMat(croppedScreenshot); // Create Mat converted from croppedScreenshot
}
}
else
{
mat = BitmapConverter.ToMat(screenshot); // Create Mat converted from screenshot Bitmap
if (mat == null) // Check if mat is null
Console.WriteLine("Could not create haystack Mat from screenshot. Image search canceled."); // Output info to log
}
}
return mat; // Return mat
}
GetDuplicationScreenshot calls your snippet. The error occurs at mat = BitmapConverter.ToMat(screenshot). I AM able to save test.png.
exergist
exergist2y ago
(for some reason Discord won't let me enter this)
exergist
exergist2y ago
@Jester well.....it's working now. Not sure what did it but i'm no longer getting errors O_o Though I will say that I do still occasionally get a black screen Also I found that the memory error I mentioned previously only occurs when I hit the mat = BitmapConverter.ToMat(screenshot) line while debugging
exergist
exergist2y ago
So TLDR - things seem to work fine now that I've made this change (I think there was some kind of bitmap sharing issue between the snippet and OpenCvSharp, so doing the CV work on a NEW version of the snippet bitmap did the trick)
exergist
exergist2y ago
BUT I do still receive black screens occasionally
exergist
exergist2y ago
Doing an "is the bitmap all black" check costs about 30-40 ms, which I might be willing to pay if the occasional black images are not avoidable (https://stackoverflow.com/questions/2556447/whats-an-efficient-way-to-tell-if-a-bitmap-is-entirely-black/2556571#2556571)
Stack Overflow
What's an efficient way to tell if a bitmap is entirely black?
I'm wondering if there's a super-efficient way of confirming that an Image object references an entirely black image, so every pixel within the bitmap is ARGB(255, 0, 0, 0). What would you recomme...
exergist
exergist2y ago
I also noticed I'm way more likely to get a black image if my mouse cursor is moving at the time of screen capture
Jester
Jester2y ago
<:PES_Think:639363477458255874>
exergist
exergist2y ago
I previously encountered the access violation depending on how I attempted to use the bitmap after it was created (I only explored this at 3440p). I was able to work around it via trial and error but I never figured out the root cause.
Accord
Accord2y ago
Was this issue resolved? If so, run /close - otherwise I will mark this as stale and this post will be archived until there is new activity.
exergist
exergist2y ago
Well @Jester what you've provided has proven to be super helpful. Thank you very much!!
Want results from more Discord servers?
Add your server
More Posts