C#•14mo ago

❔ Enum with data

I'm working on a lexer, and I'm encountering this problem where I need to associate different data with different token types. My first attempt was this:

internal enum TokenKind
{
    A,
    B,
    C
}

internal struct Token
{
    private TokenKind kind;

    // Optional properties depending on which token kind
    internal long? AData { get; }
    internal string? BData { get; }
    internal bool? CData { get; }
}

internal enum TokenKind
{
    A,
    B,
    C
}

internal struct Token
{
    private TokenKind kind;

    // Optional properties depending on which token kind
    internal long? AData { get; }
    internal string? BData { get; }
    internal bool? CData { get; }
}

then you can use it like so:

switch (token.kind)
{
    case TokenKind.A:
        // Do something with AData
        Console.WriteLine(token.AData!);
        break;
    case TokenKind.B:
        // Do something with BData
        Console.WriteLine(token.BData!);
        break;
    case TokenKind.C:
        // Do something with CData
        Console.WriteLine(token.CData!);
        break;
}

switch (token.kind)
{
    case TokenKind.A:
        // Do something with AData
        Console.WriteLine(token.AData!);
        break;
    case TokenKind.B:
        // Do something with BData
        Console.WriteLine(token.BData!);
        break;
    case TokenKind.C:
        // Do something with CData
        Console.WriteLine(token.CData!);
        break;
}

I kind of dislike this implementation though. For one it seems kind of messy with the null-forgiving stuff, but it also makes the token type very bulky since it requires a separate field for every data variant. Since there will be a ton of tokens, it's important to minimize their size. It's also error-prone in the sense that someone can try to read data without first checking the token kind. What I'm trying to do is conceptually this:

enum TokenKind {
  A(i64),
  B(String),
  C(bool),
}

enum TokenKind {
  A(i64),
  B(String),
  C(bool),
}

It doesn't have to look like this, but this is the kind of behavior I'm looking for (Rust enums, or algebraic sum types for those who know). What would be the clean way to make this in C#? Bear in mind There is actually a lot of token kinds, and not all token kinds have data associated with them (in fact the majority do not).

49 Replies

Buddy•14mo ago

Unfortunately C# does not allow for data in Enums, I personally love Rust's Enums.

__dil__OP•14mo ago

So do I, and I am aware that it's not directly possible in C#, but surely there's some way to achieve similar semantics, no? How do C# devs solve these kinds of problems?

Buddy•14mo ago

You can just use a struct called something like 'Token' with properties such as Value, Index and Type. Type property being an enum.

__dil__OP•14mo ago

I'm not sure what Index refers to here, and what would be the type of Value?

Buddy•14mo ago

Type property being an enum.

__dil__OP•14mo ago

yeah I get Type, what about the others

Buddy•14mo ago

Value being the value of the token, example (. Index refers to the index of where the token is found.

__dil__OP•14mo ago

Oh, I don't need those as my lexer works with spans. So let's set those aside for a sec. So, is there truly not a way to achieve this using polymorphism or whatever? Because that doesn't seem like it solves the problem at all, it doesn't include the type-specific data unless I'm misunderstanding

Thinker•14mo ago

This is a pattern I use a lot

public abstract record Token
{
    public sealed record Regular(TokenKind Kind) : Token;

    public sealed record Integer(TokenKind, int Value) : Token;
}

public abstract record Token
{
    public sealed record Regular(TokenKind Kind) : Token;

    public sealed record Integer(TokenKind, int Value) : Token;
}

Buddy•14mo ago

That's better. But .. Yeah.

Thinker•14mo ago

Although this won't work if you're using spans and ref structs

__dil__OP•14mo ago

(to be clear, by span I meant a struct that contains a start byte offset and an end byte offset, not like the C# Span type)

Gaming

Programming

❔ Enum with data