C#•14mo ago

❔ IL CodeGen for toy compiler [advice]

I am trying to learn more about compilers. I'd like to target some type of IR language for a toy compiler, because I don't really want to target a high-level language and I don't want to mess with LLVM stuff yet. WASM seems like a good target and I'll probably go with it if this IL stuff does not work out, but being able to leverage a bunch of BCL code from the start even for a toy language sounds really cool, so I'd like to start out targeting a small, small subset of IL. Problem is, I'm having trouble producing an assembly, like, at all: 1. I keep reading System.Reflection.Emit can write a dynamic assembly to a PE, but all this seems like it's .NET Framework stuff. I've also heard emit can be slow but this is a toy project so whatever 2. When I try to override Compilation from Microsoft.CodeAnalysis, there are internal abstract members which tells me I'm not supposed to touch it (I'm sure implementing MyLangCompilation is wayyy more work than I realize, but this seems like the nicest option) 3. (https://github.com/kkokosa/Mobius.ILasm)[Mobius.ILasm] is apparently what https://sharplab.io uses to compile IL, and while this is a toy project and I'm not stuck up, I was hoping to plug into something from dotnet organization. Seems like this is the best bet so far 4. I know the F# compiler predates Roslyn and can produce IL on it own, so I tried peeking into it to see if there are any interfaces (nuget packages, dependencies, idk, etc.,) that it plugs into when emitting IL.. you know, for things like release signing, writing to streams, byte alignment,, just "staying in spec" type stuff, and I came away so confused seeing why the Rosyln team didn't bother trying to fit in F#. 5. Maybe Mono.Cecil ????

SharpLab

C#/VB/F# compiler playground.

GitHub

GitHub - jbevain/cecil: Cecil is a library to inspect, modify and c...

Cecil is a library to inspect, modify and create .NET programs and libraries. - GitHub - jbevain/cecil: Cecil is a library to inspect, modify and create .NET programs and libraries.

21 Replies

reflectronic•14mo ago

yeah, something like Cecil is your best bet. the other options are bad ideas/do not work

reflectronic•14mo ago

personally i think Cecil is kind of a crummy library and i would suggest something like https://github.com/Washi1337/AsmResolver instead

GitHub

GitHub - Washi1337/AsmResolver: A library for creating, reading and...

A library for creating, reading and editing PE files and .NET modules. - GitHub - Washi1337/AsmResolver: A library for creating, reading and editing PE files and .NET modules.

reflectronic•14mo ago

there is also the System.Reflection.Metadata API if you would prefer to work directly with the lower-level constructs described in the ECMA-335 standard, AsmResolver is somewhat higher level

麺麭OP•14mo ago

interesting Right now I'm just looking for a "fastest up and running" that I can then tweak with to try to understand why it works i'll take a look at it and don't expect any more of your time but if you've got any micro examples for outputting something I can execute with dotnet run i'll gladly take them

reflectronic•14mo ago

here is something simple using AsmResolver

using AsmResolver.DotNet;
using AsmResolver.DotNet.Code.Cil;
using AsmResolver.DotNet.Signatures;
using AsmResolver.PE.DotNet.Cil;
using AsmResolver.PE.DotNet.Metadata.Tables.Rows;

var assembly = new AssemblyDefinition(
    name: "MyAssembly",
    version: new(1, 0, 0, 0));

var module = new ModuleDefinition("MyAssembly.exe");
var factory = module.CorLibTypeFactory;

assembly.Modules.Add(module);

var programType = new TypeDefinition(
    ns: "", 
    name: "Program", 
    TypeAttributes.Class,
    baseType: factory.Object.Type);

module.TopLevelTypes.Add(programType);

var mainMethod = new MethodDefinition(
    name: "Main",
    MethodAttributes.Public | MethodAttributes.Static,
    signature: MethodSignature.CreateStatic(returnType: module.CorLibTypeFactory.Void));

var methodBody = new CilMethodBody(mainMethod);
mainMethod.CilMethodBody = methodBody;

var writeLine = factory.CorLibScope
    .CreateTypeReference("System", "Console")
    .CreateMemberReference("WriteLine", MethodSignature.CreateStatic(
        factory.Void, factory.String))
    .ImportWith(module.DefaultImporter);

var il = methodBody.Instructions;
il.Add(CilOpCodes.Ldstr, "Hello World!");
il.Add(CilOpCodes.Call, writeLine);
il.Add(CilOpCodes.Ret);

programType.Methods.Add(mainMethod);
module.ManagedEntryPointMethod = mainMethod;

assembly.Write("MyAssembly.exe");

using AsmResolver.DotNet;
using AsmResolver.DotNet.Code.Cil;
using AsmResolver.DotNet.Signatures;
using AsmResolver.PE.DotNet.Cil;
using AsmResolver.PE.DotNet.Metadata.Tables.Rows;

var assembly = new AssemblyDefinition(
    name: "MyAssembly",
    version: new(1, 0, 0, 0));

var module = new ModuleDefinition("MyAssembly.exe");
var factory = module.CorLibTypeFactory;

assembly.Modules.Add(module);

var programType = new TypeDefinition(
    ns: "", 
    name: "Program", 
    TypeAttributes.Class,
    baseType: factory.Object.Type);

module.TopLevelTypes.Add(programType);

var mainMethod = new MethodDefinition(
    name: "Main",
    MethodAttributes.Public | MethodAttributes.Static,
    signature: MethodSignature.CreateStatic(returnType: module.CorLibTypeFactory.Void));

var methodBody = new CilMethodBody(mainMethod);
mainMethod.CilMethodBody = methodBody;

var writeLine = factory.CorLibScope
    .CreateTypeReference("System", "Console")
    .CreateMemberReference("WriteLine", MethodSignature.CreateStatic(
        factory.Void, factory.String))
    .ImportWith(module.DefaultImporter);

var il = methodBody.Instructions;
il.Add(CilOpCodes.Ldstr, "Hello World!");
il.Add(CilOpCodes.Call, writeLine);
il.Add(CilOpCodes.Ret);

programType.Methods.Add(mainMethod);
module.ManagedEntryPointMethod = mainMethod;

assembly.Write("MyAssembly.exe");

<PackageReference Include="AsmResolver.DotNet" Version="5.4.0" />

<PackageReference Include="AsmResolver.DotNet" Version="5.4.0" />

麺麭OP•14mo ago

Works first try! Thank you!

Accord•14mo ago

Was this issue resolved? If so, run /close - otherwise I will mark this as stale and this post will be archived until there is new activity.

麺麭OP•13mo ago

do you mind if I ask another question on this? I actually got some code from my language to compile to valid IL, but im trying to change the TFM to 5+ and I keep getting

Unhandled Exception: System.IO.FileNotFoundException: Could not load file or assembly 'System.Runtime, Version=5.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.

Unhandled Exception: System.IO.FileNotFoundException: Could not load file or assembly 'System.Runtime, Version=5.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.

and

reflectronic•13mo ago

if you are just running the .exe it will not work when you run an .exe that is a .NET assembly, it always runs in .NET Framework

麺麭OP•13mo ago

I've tried these methods and tried screwing around in the source code to see what it actually does, but im working in a notebook rn so i can't switch to VS and turn off Just My Code yet https://docs.washi.dev/asmresolver/guides/dotnet/basics.html#creating-a-new-net-module

reflectronic•13mo ago

the IL is almost certainly fine, you just need to run it in .NET Core instead of .NET Framework the easiest way is probably to just do dotnet MyOutput.dll it will nag you for a runtimeconfig.json, so add MyOutput.runtimeconfig.json that looks like this

{
   "runtimeOptions": {
     "tfm": "net5.0",
     "framework": {
       "name": "Microsoft.NETCore.App",
       "version": "5.0.0"
     }
   }
}

{
   "runtimeOptions": {
     "tfm": "net5.0",
     "framework": {
       "name": "Microsoft.NETCore.App",
       "version": "5.0.0"
     }
   }
}

麺麭OP•13mo ago

you are coming in as the GOAT

reflectronic•13mo ago

unless you actually have .NET 5 you will need to configure roll-forward. so, add "rollForward": "LatestMajor", after "tfm": "net5.0",

麺麭OP•13mo ago

I've published and run self-contained apps but I don't think i've actually run a framework dependent app

Unhandled exception. System.TypeLoadException: Could not load type 'System.Console' from assembly 'System.Runtime, Version=7.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.
   at Program.Main()

Unhandled exception. System.TypeLoadException: Could not load type 'System.Console' from assembly 'System.Runtime, Version=7.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.
   at Program.Main()

have you encountered this before? oh damn Console has its own assembly now or I would have assumed this would be part of corlib because it's obv not nuget? this is kinda just stream of thoughts for me so sorry not expecting u to hand-hold, i just have a lot to learn lol 😅 Oh it's Internal.Console in CoreLib? hmm

reflectronic•13mo ago

there are dozens of assemblies that come with the Microsoft.NETCore.App framework there is an Internal.Console but it's used for print debugging during development (since Console doesn't exist in CoreLib), that's not the implementation of Console

麺麭OP•13mo ago

I see comparing on sharplib that the assembly referenced is call void [System.Console]System.Console::Write(int32) I also see on the code you provided for netfx that the type is imported with the default importer i'm guessing i have to qualify it? this is probably where I should read the docs more

reflectronic•13mo ago

there was an effort to make mscorlib.dll smaller by splitting unrelated things into different assemblies, which is why there is System.Console.dll, System.Collections.dll, etc. yes, you would have to import System.Console

麺麭OP•13mo ago

I fumbled my way through 🥳

麺麭OP•13mo ago

thank you!! I do kind of have a curiosity question if ur still there tho maybe it doesn't belong in this channel? is the distinction between module and assembly an artifact of net framework days, or are there assemblies out there that have more than one module? because i think from just the average C# developer perspective, you're expected to know about assemblies and namespaces but not screw with modules much interesting https://stackoverflow.com/questions/9271805/net-module-vs-assembly i think this is the answer right here

reflectronic•13mo ago

yeah, that answer is alright in .NET Core there is no support for multi-module assemblies so, i would not try to figure out how to make them work, because they won't

Accord•13mo ago

Was this issue resolved? If so, run /close - otherwise I will mark this as stale and this post will be archived until there is new activity.

Gaming

Programming

❔ IL CodeGen for toy compiler [advice]