C#•14mo ago

✅ Modify delegate for ILGPU acceptance

This is a long one, strap in! So I’m incorporating ILGPU into a library I’m writing for very simple but powerful/expressive/flexible parallel programming. ILGPU is gonna help power the upcoming GPU API. With that said, ILGPU accepts either a delegate or a static method as a GPU kernel. The method called has to incorporate an instantiation of the ILGPU.IIndex interface as the first parameter. For the sake of my API, I would like to avoid passing around ILGPU datatypes to delegates I accept, because that exposes implementation details to users of my library. With that the background out of the way, ILGPU kernels, if they are a delegate, do not support captures or objects. This poses a problem for me, since I need to pass a delegate to the ILGPU library with an IIndex but abstract that away in my API. This problem exists with other parameters, too. What I want to be able to do is have something like:

delegate<Index1D, ArrayView<T>, void> kernel = (idx, av) =>
{
    int i = idx.X; //simple example
    var g = new GPUArray(av);
    action_received_from_user(i, g);
}

delegate<Index1D, ArrayView<T>, void> kernel = (idx, av) =>
{
    int i = idx.X; //simple example
    var g = new GPUArray(av);
    action_received_from_user(i, g);
}

but the problem here is that the kernel delegate captures action_received_from_user, which ILGPU doesn’t allow. I tried passing in a struct as one of the kernel arguments that contains action_received_from_user as a member, but because delegates are managed types, they cannot be passed in as a kernel argument. What I am thinking of is one of the following: 1. A way to on-the-fly inline (inline in the C/C++ sense) action_received_from_user into my kernel body. 2. A way to use Mono.Cecil or some other reflection API to manually inject CIL into the method body and change the argument types (I’m very familiar with CIL, less so with reflection) 3. Use Roslyn -> ??? -> Profit…? 4. Some other way to accomplish this. Phew, that was long. I’ve been trying to think of solutions for days and I have tried just about everything under the sun, but none of them have worked. Any ideas?

13 Replies

jcotton42•14mo ago

since this is parallel stuff, is it in the same address space?

InsertPiOP•14mo ago

yes, and everything going on here (on the CPU side) is only being executed by a single thread

jcotton42•14mo ago

a GCHandle or function pointer may work for you

jcotton42•14mo ago

https://learn.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.gchandle?view=net-7.0 https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/unsafe-code

GCHandle Struct (System.Runtime.InteropServices)

Provides a way to access a managed object from unmanaged memory.

Unsafe code, pointers to data, and function pointers - C#

Learn about unsafe code, pointers, and function pointers. C# requires you to declare an unsafe context to use these features to directly manipulate memory or function pointers (unmanaged delegates).

InsertPiOP•14mo ago

i looked at function pointers (just learned about them yesterday), but couldn’t quite put the pieces together into a solution

jcotton42•14mo ago

the point is, you need to get a C# delegate into ILGpu as an arg right?

InsertPiOP•14mo ago

getting a C# delegate into ILGPU is not the problem, the problem is that i want to wrap another delegate and do some bookkeeping before passing off control flow though looking at this GCStruct has given me ideas! i’ll play with it tonight and see if i cant come up with something. i wasnt aware that existed

jcotton42•14mo ago

you may also be able to just get a pointer to the delegate using & (you may also need fixed) which solution you need depends on how long the delegate needs to live

InsertPiOP•14mo ago

the big problem is that delegates passed to ILGPU as a GPU kernel are not permitted to capture values so any solution of the form

(ILGPU params) => {
    my defined params = do_bookkeeping(ILGPU params);
    action(my defined params);
}

(ILGPU params) => {
    my defined params = do_bookkeeping(ILGPU params);
    action(my defined params);
}

wont work because action is captured so update, turns out that ILGPU doesn’t support .NET pointer types meaning that function pointers and GCHandle structs won’t work

jcotton42•14mo ago

Does it support IntPtr? Or, if you can assume x64 arch, long?

InsertPiOP•14mo ago

Accord•14mo ago

Was this issue resolved? If so, run /close - otherwise I will mark this as stale and this post will be archived until there is new activity.

InsertPiOP•14mo ago

I have found a solution! Digging through ILGPU I found some options for “explicitly grouped kernels,” which let me do what I wanted to do. I didn’t think those existed, and I thought I’d end up with some bizarre C# code which is why I posted here. Anyway, it’s resolved now

Gaming

Programming

✅ Modify delegate for ILGPU acceptance