C#•2y ago

✅ ✅ Vararg P/Invoke (x86) throws BadImageFormatException (0x80131124 "Index not found")

I am attempting to perform vararg P/Invokes following signatures I see on pinvoke.net and, e.g., https://stackoverflow.com/questions/2124490/what-is-the-proper-pinvoke-signature-for-a-function-that-takes-var-args This led me to the following P/Invokes:

[DllImport("msvcrt.dll", CharSet = CharSet.Ansi, CallingConvention = CallingConvention.Cdecl)]
public static unsafe extern int sprintf(IntPtr buffer, string format, __arglist);

[DllImport("msvcrt.dll", CharSet = CharSet.Ansi, CallingConvention = CallingConvention.Cdecl)]
public static unsafe extern int _scprintf(string format, __arglist);

[DllImport("msvcrt.dll", CharSet = CharSet.Ansi, CallingConvention = CallingConvention.Cdecl)]
public static unsafe extern int sprintf(IntPtr buffer, string format, __arglist);

[DllImport("msvcrt.dll", CharSet = CharSet.Ansi, CallingConvention = CallingConvention.Cdecl)]
public static unsafe extern int _scprintf(string format, __arglist);

invoked as such:

public static void CLRPrintfHook(string fmt, nint va0, nint va1, // ...
// {...}
_scprintf(fmt, __arglist(va0, va1));

public static void CLRPrintfHook(string fmt, nint va0, nint va1, // ...
// {...}
_scprintf(fmt, __arglist(va0, va1));

specifically for

<PlatformTarget>x86</PlatformTarget>

<PlatformTarget>x86</PlatformTarget>

using .NET 7. But they result in a BadImageFormatException with result 0x80131124, "Index not found." See image. I do not have problems issuing P/Invokes to, e.g. _vscprintf and vsprintf which take va_list instead of varargs as the final argument. For posterity, the working P/Invokes are declared as:

    [LibraryImport("msvcrt.dll", StringMarshalling = StringMarshalling.Custom, StringMarshallingCustomType = typeof(System.Runtime.InteropServices.Marshalling.AnsiStringMarshaller))]
    [UnmanagedCallConv(CallConvs = new Type[] { typeof(System.Runtime.CompilerServices.CallConvCdecl) })]
    public static partial int _vscprintf(string format, IntPtr ptr);

    [LibraryImport("msvcrt.dll", StringMarshalling = StringMarshalling.Custom, StringMarshallingCustomType = typeof(System.Runtime.InteropServices.Marshalling.AnsiStringMarshaller))]
    [UnmanagedCallConv(CallConvs = new Type[] { typeof(System.Runtime.CompilerServices.CallConvCdecl) })]
    public static partial int vsprintf(IntPtr buffer, string format, IntPtr args);

    [LibraryImport("msvcrt.dll", StringMarshalling = StringMarshalling.Custom, StringMarshallingCustomType = typeof(System.Runtime.InteropServices.Marshalling.AnsiStringMarshaller))]
    [UnmanagedCallConv(CallConvs = new Type[] { typeof(System.Runtime.CompilerServices.CallConvCdecl) })]
    public static partial int _vscprintf(string format, IntPtr ptr);

    [LibraryImport("msvcrt.dll", StringMarshalling = StringMarshalling.Custom, StringMarshallingCustomType = typeof(System.Runtime.InteropServices.Marshalling.AnsiStringMarshaller))]
    [UnmanagedCallConv(CallConvs = new Type[] { typeof(System.Runtime.CompilerServices.CallConvCdecl) })]
    public static partial int vsprintf(IntPtr buffer, string format, IntPtr args);

Any idea as to what I could be doing wrong? Or is this plain unsupported?

43 Replies

Petris•2y ago

@jkortech might know the best here

SingleAccretion•2y ago

Possible this is a runtime bug.

peppyOP•2y ago

What do you need me to provide to investigate? Happy to share the entire thing.

SingleAccretion•2y ago

Well, perhaps I am not quite the right person to investigate, if you wanted to take a deeper look yourself, you could obtain symbols for CoreCLR / build it in Debug and see what's throwing the exception. FWIW, I just tried this with a .NET8 runtime build in a hello world setting, and it did not crash.

peppyOP•2y ago

Yeah I'm trying to construct a minimal repro as well, will update as soon as I have Yes. A minimal repro (.NET 7, x86, just the P/Invokes as stated above and a naive invocation) seem to work fine. Which means I should better explain the fine print which I had hoped to avoid. I thought it was some trivial mistake on my behalf.

// abridged for brevity...
string fmt = "heap: %x-%x"; nint n0 = 0x0ecc1f00; nint n1 = 0x0fdc1f00;
int    a   = _scprintf(fmt, __arglist(n0, n1));
nint   buf = Marshal.AllocHGlobal(bl);
int    b   = sprintf(b, fmt, __arglist(n0, n1));
string s   = Marshal.PtrToStringAnsi(buf);
// works as intended!

// abridged for brevity...
string fmt = "heap: %x-%x"; nint n0 = 0x0ecc1f00; nint n1 = 0x0fdc1f00;
int    a   = _scprintf(fmt, __arglist(n0, n1));
nint   buf = Marshal.AllocHGlobal(bl);
int    b   = sprintf(b, fmt, __arglist(n0, n1));
string s   = Marshal.PtrToStringAnsi(buf);
// works as intended!

ero•2y ago

what's the use case of calling sprintf here by the way?

peppyOP•2y ago

In essence, I am detouring/hooking a void dbgPrintf(const char* fmt, ...); call in a x86 C++ executable. In the initial post _scprintf(fmt, __arglist(va0, va1)); is part of a hook I install by Detours in a process I'm interested in. Basically took https://github.com/citronneur/detours.net and blended it with the MSDN .NET hosting sample. It already gave me some grief earlier today (https://discord.com/channels/143867839282020352/1115271191310110840/1115271191310110840), so I'm by no means in familiar waters. Stupidity on my end is absolutely a given in this scenario. To get around the fact the original function's signature is (char* fmt, ...) as opposed to, say, (char* fmt, va_args argptr) I try declaring:

public static void naive_printf_hook(string fmt, nint va0, nint va1, nint va2, nint va3 // ...);

public static void naive_printf_hook(string fmt, nint va0, nint va1, nint va2, nint va3 // ...);

and depending on the actual number of format specifiers given in fmt I declare the appropriate __arglist, e.g. __arglist(va0, va1)). I had hoped the original function would take va_args as its final parameter because then I could just use some vsprintf type call (on x86 Windows, va_args argptr should be equivalent to nint argptr in a P/Invoke signature, AFAIK?), but no dice. Hence faffing with _scprintf and sprintf. Don't get me wrong; I know this is stupid, but as stupid as it is, it should be possible in this way or another; I'm just at a loss as to why this specific exception in this specific case

SingleAccretion•2y ago

Well - it does not look possible to guess to me, so it would need to be debugged.

peppyOP•2y ago

I've just enabled the MS symbol server in VS so I'll see if they take me somewhere. I've never debugged something this... obtuse before, so it'll take me a bit to get up to speed.

peppyOP•2y ago

So I've arrived at the following trace using the pure native debugger.

peppyOP•2y ago

Would providing the .dmp help in this case? I, bluntly put, have no idea what I'm looking at here. I would just hand in the whole project, but this being a game hook, "reproducible" and "minimal" would be very hard to satisfy. (Well, more the latter than the former...) For posterity this is VS '22 17.6.2 Written out for easier grepping than the image:

KERNELBASE.dll!_RaiseException@16()
coreclr.dll!RaiseTheExceptionInternalOnly(Object * throwable, int rethrow, int) Line 2806
    at D:\a\_work\1\s\src\coreclr\vm\excep.cpp(2806)
coreclr.dll!UnwindAndContinueRethrowHelperAfterCatch(Frame * pException, Exception *) Line 7758
    at D:\a\_work\1\s\src\coreclr\vm\excep.cpp(7758)
coreclr.dll!GetILStubForCalli(VASigCookie * pVASigCookie, MethodDesc * pMD) Line 6072
    at D:\a\_work\1\s\src\coreclr\vm\dllimport.cpp(6072)
coreclr.dll!VarargPInvokeStubWorker(TransitionBlock * pTransitionBlock, VASigCookie * pVASigCookie, MethodDesc * pMD) Line 5920
    at D:\a\_work\1\s\src\coreclr\vm\dllimport.cpp(5920)
coreclr.dll!_VarargPInvokeStub@0()
056c61c0()
[Frames below may be incorrect and/or missing]
056c5e33()
FFX.exe!0062c301()
FFX.exe!0062c2c9()
FFX.exe!006125ae()
FFX.exe!004265d3()
FFX.exe!004f3a3c()
kernel32.dll!@BaseThreadInitThunk@12()
ntdll.dll!___RtlUserThreadStart@8()
ntdll.dll!__RtlUserThreadStart@8()

KERNELBASE.dll!_RaiseException@16()
coreclr.dll!RaiseTheExceptionInternalOnly(Object * throwable, int rethrow, int) Line 2806
    at D:\a\_work\1\s\src\coreclr\vm\excep.cpp(2806)
coreclr.dll!UnwindAndContinueRethrowHelperAfterCatch(Frame * pException, Exception *) Line 7758
    at D:\a\_work\1\s\src\coreclr\vm\excep.cpp(7758)
coreclr.dll!GetILStubForCalli(VASigCookie * pVASigCookie, MethodDesc * pMD) Line 6072
    at D:\a\_work\1\s\src\coreclr\vm\dllimport.cpp(6072)
coreclr.dll!VarargPInvokeStubWorker(TransitionBlock * pTransitionBlock, VASigCookie * pVASigCookie, MethodDesc * pMD) Line 5920
    at D:\a\_work\1\s\src\coreclr\vm\dllimport.cpp(5920)
coreclr.dll!_VarargPInvokeStub@0()
056c61c0()
[Frames below may be incorrect and/or missing]
056c5e33()
FFX.exe!0062c301()
FFX.exe!0062c2c9()
FFX.exe!006125ae()
FFX.exe!004265d3()
FFX.exe!004f3a3c()
kernel32.dll!@BaseThreadInitThunk@12()
ntdll.dll!___RtlUserThreadStart@8()
ntdll.dll!__RtlUserThreadStart@8()

jkortech•2y ago

Any chance you can enable first-chance exception handling for C++ exceptions? This is catching the re-throw of the exception as a managed exception.

peppyOP•2y ago

How can I do this?

jkortech•2y ago

In the ExceptionSettings window, there's a section called C++ Exceptions. Check that box while debugging In WinDBG, there's a settings page for exceptions, or you can run sxe eh in the command window

peppyOP•2y ago

(23d8.f80): C++ EH exception - code e06d7363 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
eax=3fe7f9e0 ebx=19930520 ecx=00000003 edx=00000000 esi=80131124 edi=66f4add4
eip=76bc8462 esp=3fe7f9e0 ebp=3fe7fa3c iopl=0         nv up ei pl nz ac po nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000212
KERNELBASE!RaiseException+0x62:
76bc8462 8b4c2454        mov     ecx,dword ptr [esp+54h] ss:002b:3fe7fa34=abf242e7

(23d8.f80): C++ EH exception - code e06d7363 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
eax=3fe7f9e0 ebx=19930520 ecx=00000003 edx=00000000 esi=80131124 edi=66f4add4
eip=76bc8462 esp=3fe7f9e0 ebp=3fe7fa3c iopl=0         nv up ei pl nz ac po nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000212
KERNELBASE!RaiseException+0x62:
76bc8462 8b4c2454        mov     ecx,dword ptr [esp+54h] ss:002b:3fe7fa34=abf242e7

Is this right? I've never used WinDbg before, but I think I should be able to catch it. Not sure where to go from here to get you the details you need.

peppyOP•2y ago

For posterity, this is the call stack I get if I stop when I receive the message given above:

peppyOP•2y ago

It's different this time- maybe this is what you were looking for? (I did nothing but sxe eh and let it run until it threw, then opened the call stack display.)

peppyOP•2y ago

With source addresses:

peppyOP•2y ago

It's getting very late here, so I'll be off. Please let me know if I can provide any other info of interest, I'll do it first thing in the morning. Thank you in advance for your time and help.

jkortech•2y ago

That's the information I was looking for, but the stack trace is definitely weird Is this a release build/shipped .NET or is this a debug build? A dump with the debugger at this point would be helpful.

peppyOP•2y ago

Uh- it's a debug build of the application itself, and the .NET SDK is whatever comes with VS '22 17.6.2

ero•2y ago

There's a lot that come with that :p Could be 6, 7, or 8

peppyOP•2y ago

In this instance specifically .NET 7.

PS C:\Users\kfr> dotnet --list-sdks
7.0.302 [C:\Program Files\dotnet\sdk]

PS C:\Users\kfr> dotnet --list-sdks
7.0.302 [C:\Program Files\dotnet\sdk]

What dump options should I specify in WinDbg? .dump /ma {...} is available, but it is 350+MB in size

peppyOP•2y ago

Here is the result of .dump /ma. https://we.tl/t-ci3mqsGrpl

dump.dmp

1 file sent via WeTransfer, the simplest way to send your files around the world

peppyOP•2y ago

If I should use different switches or you need any supplementary data don't hesitate to let me know.

jkortech•2y ago

I've made some progress with this So, the call that's actually throwing the exception is on line 4283. It looks like your metadata image is corrupted. Do you run any post-processing tools on your assembly? I'll see if I can figure out more.

peppyOP•2y ago

Post-processing? What would fall into 'post-processing'? The assembly containing the P/Invoke signature is just a regular old class library: https://github.com/fkelava/fahrenheit/blob/main/src/cs/Fahrenheit.CoreLib/Fahrenheit.CoreLib.csproj and the assembly using it doesn't seem to me to be out of the ordinary either: https://github.com/fkelava/fahrenheit/blob/main/src/cs/Fahrenheit.CLRHost/Fahrenheit.CLRHost.csproj I build them and that's it- I host the CLR and jump into a static method in CLRHost The "special" thing in this instance is that I'm hosting the CLR, but then again, any non-vararg P/Invokes work fine in the exact same scenario all the way in the original post, for instance, there are _vscprintf and vsprintf signatures that do not exhibit any issues

jkortech•2y ago

I think I found the issue (and it might actually be a bug in the runtime)

Petris•2y ago

(good thing that I've pinged you then when

)

jkortech•2y ago

Is the call to the vararg P/Invoke in the same assembly as the P/Invoke, or is the call to it in fhcshook.dll?

peppyOP•2y ago

The call is in fhcshook.dll. In fact, in my use case the call will never be in the same assembly as the P/Invoke.

jkortech•2y ago

Can you try changing your code temporarily to make the call happen in fhcorelib.dll instead of in fhcshook.dll, just for this one test?

peppyOP•2y ago

I assume this also means I can just place the P/Invoke definitions in fhcshook, or does it specifically have to be what you just said

jkortech•2y ago

That also would work

peppyOP•2y ago

Sure, one moment

jkortech•2y ago

The caller and callee just need to be in the same assembly

peppyOP•2y ago

Got it, be right back the _scprintf call indeed does not crash if the P/Invoke is declared in fhcshook itself seems to work as intended now

1686077931073 | [Info] printfh.cs:83 (CLRPrintfHook): fmt: Virtuos: Loaded %s successfully, size = %d
1686077931073 | [Info] printfh.cs:84 (CLRPrintfHook): argc: 2
1686077931073 | [Info] printfh.cs:108 (CLRPrintfHook): bl: 94

1686077931073 | [Info] printfh.cs:83 (CLRPrintfHook): fmt: Virtuos: Loaded %s successfully, size = %d
1686077931073 | [Info] printfh.cs:84 (CLRPrintfHook): argc: 2
1686077931073 | [Info] printfh.cs:108 (CLRPrintfHook): bl: 94

etc. etc.

jkortech•2y ago

Okay, I'll file a bug Can't guarantee we'll fix it for .NET 8 though

peppyOP•2y ago

I appreciate the help regardless- if you could just post a link so I can track the issue, that'd be great Thank you very much, I was in over my head here

jkortech•2y ago

Here's the issue: https://github.com/dotnet/runtime/issues/87188

GitHub

Calling a Vararg P/Invoke from another assembly fails to call under...

Description When calling a vararg P/Invoke defined in another assembly, the runtime tries to look up the metadata from the wrong module. This can cause a variety of issues, from throwing a BadImage...

peppyOP•2y ago

Glad it was discovered. Thank you once again for your time and patience. The workaround is perfectly fine for now; it works well.

MODiX•2y ago

Use the /close command to mark a forum thread as answered

Accord•2y ago

Was this issue resolved? If so, run /close - otherwise I will mark this as stale and this post will be archived until there is new activity.

Gaming

Programming

✅ ✅ Vararg P/Invoke (x86) throws BadImageFormatException (0x80131124 "Index not found")