C#•2y ago

❔ System.Linq.Expressions custom expression

Hi guys! I'm parsing a custom language and building an Expression Tree based on it, but I came into an issue: My language can use Labels, but since I parse the file line by line, I can't reference a label that hasn't been parsed yet when creating a Goto expression. So parsing something like that would give issues:

IFEQUALS var1, 10;
  GOTO label;
END;
label:
  PRINT "HELLO WORLD";

IFEQUALS var1, 10;
  GOTO label;
END;
label:
  PRINT "HELLO WORLD";

So, my idea was to inherit the Expression class to create a kind of LazyGotoExpression, where the LabelTarget would be evaluated by the time the expression is compiled, but I have no clue how to do that. Could you guys give me a hand? Current parsing code:

  private Expression ParseExpression(List<string> tokens, Func<List<string>> parseNextStatement)
        {
            switch (tokens[0])
            {
                case "LABEL":
                    {
                        var label = tokens[1];
                        var labelTarget = Expression.Label(label);
                        _labels.Add(label, labelTarget);
                        return Expression.Label(labelTarget);
                    }
                case "GOTO":
                    {
                        //I MIGHT NOT HAVE THE LABEL REFERENCE HERE, SO I NEED A WAY TO LAZY EVALUATE IT
                        var label = tokens[1];
                        var expression = Expression.Empty();
                        _goto.Add(expression, label);
                        return expression;
                    }

  private Expression ParseExpression(List<string> tokens, Func<List<string>> parseNextStatement)
        {
            switch (tokens[0])
            {
                case "LABEL":
                    {
                        var label = tokens[1];
                        var labelTarget = Expression.Label(label);
                        _labels.Add(label, labelTarget);
                        return Expression.Label(labelTarget);
                    }
                case "GOTO":
                    {
                        //I MIGHT NOT HAVE THE LABEL REFERENCE HERE, SO I NEED A WAY TO LAZY EVALUATE IT
                        var label = tokens[1];
                        var expression = Expression.Empty();
                        _goto.Add(expression, label);
                        return expression;
                    }

If there is a way to make a custom Expression return another Expression instance, that would be perfect.

67 Replies

rick_o_maxOP•2y ago

Nvm, I guess this does the trick:

    public class LazyGotoExpression : Expression
    {
        public GameAction GameAction;
        public string Label;
        public override ExpressionType NodeType => ExpressionType.Default;

        protected override Expression VisitChildren(ExpressionVisitor visitor)
        {
            if (GameAction.Labels.TryGetValue(Label, out var labelTarget))
            {
                return Expression.Goto(labelTarget);
            }
            return Expression.Empty();
        }
    }

    public class LazyGotoExpression : Expression
    {
        public GameAction GameAction;
        public string Label;
        public override ExpressionType NodeType => ExpressionType.Default;

        protected override Expression VisitChildren(ExpressionVisitor visitor)
        {
            if (GameAction.Labels.TryGetValue(Label, out var labelTarget))
            {
                return Expression.Goto(labelTarget);
            }
            return Expression.Empty();
        }
    }

rotor_•2y ago

This is a cool looking project. Would you mind sharing the source code? I always get excited by custom languages

rick_o_maxOP•2y ago

I'll surely do. I'm porting an old Brazilian game to Unity 3D The game is Alien Anarchy I'll post the source on GitHub soon (what I've done so far)

rotor_•2y ago

Oh man that's even more cool

rick_o_maxOP•2y ago

Original game screenshot:

rick_o_maxOP•2y ago

Map loaded in Unity

rick_o_maxOP•2y ago

Still missing some floor and ceiling stuff, and I have to adjust the skybox texture So, the engine is ACKNEX (3D Game Studio). Version 3.0, I guess It uses a custom script language (similar to Basic) So I'm converting the script files to Expression Trees in runtime

rotor_•2y ago

Aaah so you're making a parser that you can plug the existing scripts into

rick_o_maxOP•2y ago

Yes There are other few games made using the same engine, so the code could potentially be used to run them in the future as well

rotor_•2y ago

Ah do you plan to keep the ported engine and this game separate, then, so other people can port whatever games they want?

rick_o_maxOP•2y ago

Yes. Idk if it is legal to share the game files. They are listed as abandonware, since the company has ended I can reach one of the developers and ask if he agrees on sharing the game data files, tho

rotor_•2y ago

Not too far off what started the FOSS movement in the first place, tbh

rick_o_maxOP•2y ago

The company who created the engine still exists, tho. And the game has some script files that company has written. One more reason I'm not sure about sharing the game files (which you can easily find anywhere in the internet) I always wanted a modern port of this game

rick_o_maxOP•2y ago

https://www.youtube.com/watch?v=YU0AWYGAeKI

Civvie 11

YouTube

The Varginha Incident - Vicious Cycles

Possibly I've seen too much. Patrons see episodes early: https://www.patreon.com/civvie11 Twitter: https://twitter.com/civvie11
Join the Dungeon Discord: https://discord.gg/CEBKKJS Links: https://www.youtube.com/user/SpeedySPCFan/videos - Speedy's channel #FreeCivvie #DOS #Retrogaming

rick_o_maxOP•2y ago

This custom expression thing I did is not working, tho

rotor_•2y ago

What's wrong with it?

rick_o_maxOP•2y ago

It doesn't seem to do what I want it to do I just want to make a Goto Expression that only evaluates when it is compiled Bc by that time, I would have the Label references created A lazy goto

rotor_•2y ago

The second snippet you pasted never sets Label Nor does it inherit it

rick_o_maxOP•2y ago

By Label, I mean any label I create by code

rick_o_maxOP•2y ago

message.txt

rick_o_maxOP•2y ago

A GOTO statement can be declared before a LABEL statement So the goto expression must be evaluated when the syntax is compiled or right before it The syntax is compiled at CloseAndCompile

rotor_•2y ago

Yes I remember bumping into a similar problem when I was writing a language parser in lua a few years back

rick_o_maxOP•2y ago

rotor_•2y ago

Random question for thought - are the labels meant to be case sensitive?

rick_o_maxOP•2y ago

No idea, in the original format they are declared like this:

//////////////////////////////// Erdbeben
    IF_EQUAL        richter,0;
        GOTO    no_quake;
       RULE    PLAYER_X = PLAYER_X + richter*(RANDOM - 0.5);
       RULE    PLAYER_Y = PLAYER_Y + richter*(random_1 - 0.5);
       RULE    PLAYER_VX = PLAYER_VX + 0.2*richter*(RANDOM - 0.5);
       RULE    PLAYER_VY = PLAYER_VY + 0.2*richter*(random_1 - 0.5);
    RULE    PLAYER_Z = PLAYER_Z + richter*(random_2 - 0.5);

    SET     random_2,random_1;
    SET     random_1,RANDOM;
no_quake:

//////////////////////////////// Erdbeben
    IF_EQUAL        richter,0;
        GOTO    no_quake;
       RULE    PLAYER_X = PLAYER_X + richter*(RANDOM - 0.5);
       RULE    PLAYER_Y = PLAYER_Y + richter*(random_1 - 0.5);
       RULE    PLAYER_VX = PLAYER_VX + 0.2*richter*(RANDOM - 0.5);
       RULE    PLAYER_VY = PLAYER_VY + 0.2*richter*(random_1 - 0.5);
    RULE    PLAYER_Z = PLAYER_Z + richter*(random_2 - 0.5);

    SET     random_2,random_1;
    SET     random_1,RANDOM;
no_quake:

I add a token LABEL as the first token in the list to indicate it is a label (only bc I'm lazy to fix it the right way) When I find something like no_quake: Nice reading

rick_o_maxOP•2y ago

https://nejcskofic.github.io/2017/07/30/extending-linq-expressions/

Code with the wind

Extending LINQ Expressions

In the previous post we looked into what expression trees are and how they can be used. While this is enough most of the time, you may find yourself wondering if you can extend expressions. This might especially be the case if you are constructing expression trees directly via factory methods and you just need to pass some additional data which ...

rick_o_maxOP•2y ago

This seems to work

    public class LazyGotoExpression : Expression
    {
        public GameAction GameAction;
        public string Label;

        public override ExpressionType NodeType => ExpressionType.Extension;

        public override Type Type => typeof(void);

        public override bool CanReduce => true;

        public override Expression Reduce()
        {
            if (GameAction.LabelsByName.TryGetValue(Label, out var labelTarget))
            {
                return Expression.Goto(labelTarget);
            }
            return Expression.Empty();
        }
    }

    public class LazyGotoExpression : Expression
    {
        public GameAction GameAction;
        public string Label;

        public override ExpressionType NodeType => ExpressionType.Extension;

        public override Type Type => typeof(void);

        public override bool CanReduce => true;

        public override Expression Reduce()
        {
            if (GameAction.LabelsByName.TryGetValue(Label, out var labelTarget))
            {
                return Expression.Goto(labelTarget);
            }
            return Expression.Empty();
        }
    }

rotor_•2y ago

Hmm you think you were overriding the wrong method before? Sorry I was still absorbing your code / watching that video you posted haha

rick_o_maxOP•2y ago

Yes, now it works, but my "if" parsing is wrong, trying to understand why I can send u the github linq later, if u want to check it out

rotor_•2y ago

I was just looking into Linq Expressions - I've never used those before. They're quite neat

rick_o_maxOP•2y ago

yos

rotor_•2y ago

Oh you might also be interested to know that switch is better than it used to be and that ParseExpression can use list patterns

var lst = new List<string> { "LABEL", "label_name", "third-argument" };
var mutable = "nothing";

switch (lst)  
  {                                                                                                                                                                
      case ["LABEL", var label_name, ..]:                                                                                                                          
          mutable = "yeh";                                                                                                                                         
          break;                                                                                                                                                   
      case ["GOTO", var label_name, ..]:                                                                                                                           
          mutable = "sure";                                         
          break;                                                    
      default:                                                      
          mutable = "nah";                                          
          break; 
  }

var lst = new List<string> { "LABEL", "label_name", "third-argument" };
var mutable = "nothing";

switch (lst)  
  {                                                                                                                                                                
      case ["LABEL", var label_name, ..]:                                                                                                                          
          mutable = "yeh";                                                                                                                                         
          break;                                                                                                                                                   
      case ["GOTO", var label_name, ..]:                                                                                                                           
          mutable = "sure";                                         
          break;                                                    
      default:                                                      
          mutable = "nah";                                          
          break; 
  }

Here's an example I've just put together

rick_o_maxOP•2y ago

Yes, I see Just could not optimize it yet I'm still getting used to newer C# syntax sugar hmm Just realized the script version the game uses has onliner IFs only Not block ifs Easier to parse

rotor_•2y ago

Your examples of IFEQUALS seem to have their body on the following line, though?

rick_o_maxOP•2y ago

The original script files doesnt

    IF_EQUAL HIT_DIST, 0; GOTO shoot2;
    IF_EQUAL HIT.FRAGILE,1; GOTO contHit;

    IF_EQUAL HIT_DIST, 0; GOTO shoot2;
    IF_EQUAL HIT.FRAGILE,1; GOTO contHit;

fire:  
    SET     LAYERS.13, Mp5_02Ovl;
    IF_EQUAL AMMO,0;
        goto noAmmo;

fire:  
    SET     LAYERS.13, Mp5_02Ovl;
    IF_EQUAL AMMO,0;
        goto noAmmo;

    IF_BELOW deathCounter, 48;
        END;

    IF_BELOW deathCounter, 48;
        END;

ACTION FlashOut{
    ADD redValue, 0.1;
    FADE_PAL pal_flash, redValue;
    IF_BELOW redValue, hitValue;
        END;

ACTION FlashOut{
    ADD redValue, 0.1;
    FADE_PAL pal_flash, redValue;
    IF_BELOW redValue, hitValue;
        END;

Just oneliners, so far Trying to understand what they call RULES

start:
//////////////////////////////// PLAYER_SIZE setzen
    IF_EQUAL        moving, Mode_Gehen;             // Gehen
        RULE    PLAYER_SIZE = my_size + 0.15*WALK;

start:
//////////////////////////////// PLAYER_SIZE setzen
    IF_EQUAL        moving, Mode_Gehen;             // Gehen
        RULE    PLAYER_SIZE = my_size + 0.15*WALK;

Seem to be expressions assigned to variables

rick_o_maxOP•2y ago

rotor_•2y ago

Strange name to choose

rick_o_maxOP•2y ago

U saw nothing yet :V There arent many keywords, tho Which is good

    IF_EQUAL        moving, Mode_Gehen;
        BRANCH  set_walking;

    IF_EQUAL        moving, Mode_Gehen;
        BRANCH  set_walking;

no idea what is branch Bc there is the CALL command as well Docs doesnt make it clear

rotor_•2y ago

There's IF and BRANCH o.O

rick_o_maxOP•2y ago

ye From the initial scripts, only 40 keywords left to "parse" Actually, just dummy methods are implemented for the keywords

rick_o_maxOP•2y ago

rick_o_maxOP•2y ago

[SIN][SQRT], etc are parsed wrongly, I guess, they are meant to be inside a RULE Other ones are generating valid expressions

rick_o_maxOP•2y ago

rotor_•2y ago

That makes sense

rick_o_maxOP•2y ago

rick_o_maxOP•2y ago

Original

rotor_•2y ago

Are you making new textures rather than importing the old ones, then?

rick_o_maxOP•2y ago

Importing the old ones

LPeter1997•2y ago

My language can use Labels, but since I parse the file line by line, I can't reference a label that hasn't been parsed yet when creating a Goto expression.

So! This is typically done by passing through the source code in multiple phases. Initially you just parse and construct a tree where you don't resolve anything. In the next phase you can start resolving entities. The problem is that the expression tree API generally expects you know everything beforehand.

rick_o_maxOP•2y ago

Yes. I fixed that by inheriting an Expression, and returning a goto statement in expression compile time

MODiX•2y ago

rick_o_max#7424

    public class LazyGotoExpression : Expression
    {
        public GameAction GameAction;
        public string Label;

        public override ExpressionType NodeType => ExpressionType.Extension;

        public override Type Type => typeof(void);

        public override bool CanReduce => true;

        public override Expression Reduce()
        {
            if (GameAction.LabelsByName.TryGetValue(Label, out var labelTarget))
            {
                return Expression.Goto(labelTarget);
            }
            return Expression.Empty();
        }
    }

    public class LazyGotoExpression : Expression
    {
        public GameAction GameAction;
        public string Label;

        public override ExpressionType NodeType => ExpressionType.Extension;

        public override Type Type => typeof(void);

        public override bool CanReduce => true;

        public override Expression Reduce()
        {
            if (GameAction.LabelsByName.TryGetValue(Label, out var labelTarget))
            {
                return Expression.Goto(labelTarget);
            }
            return Expression.Empty();
        }
    }

Quoted by

<@!934156543555932201> from #System.Linq.Expressions custom expression (click here)

React with ❌ to remove this embed.

LPeter1997•2y ago

Yeah that's actually a very clever trick I didn't know the Expression API would allow for something like this

rick_o_maxOP•2y ago

So Im thinking about the state machines now So an action (method) could be stopped/resumed

LPeter1997•2y ago

Essentially coroutines I assume

rick_o_maxOP•2y ago

I could use a switch on the methods first expressions Yes But cant use default Unity coroutines bc expressions cant yield anything Im using expressions bc I dont want to create an interpreter And code compilation does not work on iOS and WebGL on Unity

LPeter1997•2y ago

Well a wild idea, if this is a serious attempt at porting the engine, why not write a bytecode interpreter?

rick_o_maxOP•2y ago

Hmm What would be the benefits?

LPeter1997•2y ago

You don't need to do workaround with the expression tree API and it's a bit easier to debug a visible bytecode than a tree IME The downside is that it's more work yes, but if the language is fairly simple, it's not even a gigantic task I assume there's no fancy stuff like type-inference, closures, ... so it'd be a fairly simple one

rick_o_maxOP•2y ago

The language is basically an early basic version The version Alien Anarchy uses is even simpler

LPeter1997•2y ago

So not even statically typed, that's really cool Do you have some specification for the language? Or at least sizable examples?

rick_o_maxOP•2y ago

It only has one liner IFs, no LOOPs (so far, Im still going thru all sources) Will send it, one sec

rick_o_maxOP•2y ago

This is the official API doc

ACKMAN.PDF

rick_o_maxOP•2y ago

It is from a newer engine version, but many things can be applied to Alien Anarchy Sent you a PM ACTION keyword declares an action (a method) RULE is basically a local variable that receives an expression value SYNONYM represents a "group", which can have custom fields. This synonym can be applied to any game object, so N objects can use the fields from a single synonym SKILL represents a global variable (float type), which can have its values clamped automatically (min, max) IFDEF, IFNDEF, ENDIF, IFELSE, DEFINE, UNDEF are compiler directives ACTOR is used to represent an object that can move, have animations, etc THING is a static object BMAP is a bitmap (PCX files in Alien Anarchy) TEXTURE is a texture which can use a series of BITMAPs, and can be applied to walls, ceilings and floors REGION is a convex 2D region outline (topdown) WALL represents the walls between two regions WAY represents a path, made of waypoints SOUND represents a sound file (WAV in Alien Anarchy)

LPeter1997•2y ago

Yeah this looks really-really simplistic so far fortunately

rick_o_maxOP•2y ago

Script-wise, most complicated stuff are the WAIT and WAITT keywords As I said, these makes the script waits for N frames or N ticks Actions can't use parameters or return values, which makes it even simpler

rick_o_maxOP•2y ago

Waypoint parsing

Accord•2y ago

Was this issue resolved? If so, run /close - otherwise I will mark this as stale and this post will be archived until there is new activity.

Gaming

Programming

❔ System.Linq.Expressions custom expression