Topics

C#•2y ago

❔ Serialize objects from non-standard format

Hello, I am trying to serialize objects from a non-standard format and dont know how to go about it without it being complete regex or string splitting hell, any tips? The data is in text files is formatted accordingly (not all properties are always present and sub properties have variable amount of properties and can be nested): obj_name_1 = { prop_1 = 129864 prop_2 = hello prop_3 = { other_prop = bye other_prop = 56 } prop_4 = 69 } obj_name_2 = ... Thanks

63 Replies

Pacyfist•2y ago

serialize or deserialize? Also please paste code as a code block using ```csharp

Pacyfist•2y ago

also try https://code-maze.com/csharp-deserialize-json-into-dynamic-object/

Code Maze

How to Deserialize JSON Into Dynamic Object in C# - Code Maze

Describe how to deserialize JSON into dynamic object in C# with detail explanation and examples using native and Newtonsoft library

Vi Ness•2y ago

If the lines in your datafile are always formatted this nicely, you could read it line by line and check for the lines ending in { and } for your nested properties. Otherwise it's likely string splitting hell for you

ShadowTrolllOP•2y ago

There should be no wrenches being thrown in the gears, but the formatting is not always the. For example, following is possible:

obj_name_1 = {
prop_1 = {
subprop_1 = 5
subprop_2 = 6
}
}

obj_name_1 = {
prop_1 = {
subprop_1 = 5
subprop_2 = 6
}
}

obj_name_1 = {
prop_1 = { subprop_1 = 5
subprop_2 = 6
}
}

obj_name_1 = {
prop_1 = { subprop_1 = 5
subprop_2 = 6
}
}

obj_name_1 = {
prop_1 = {subprop_1 = 5
subprop_2 = 6}
}

obj_name_1 = {
prop_1 = {subprop_1 = 5
subprop_2 = 6}
}

also properties can be array of stuff, so like

obj_name_1 = {
prop_1 = {"Hello" "Bye" }
}

obj_name_1 = {
prop_1 = {"Hello" "Bye" }
}

Vi Ness•2y ago

Welp

ShadowTrolllOP•2y ago

In the end, I want both, but I want from text to object right now But my formatting is not json is it?

Vi Ness•2y ago

The arrays also using { and } is pretty nasty. Do you have any control over how these datafiles get written?

ShadowTrolllOP•2y ago

Sadly not, I just gotta take stuff I get

Vi Ness•2y ago

In your last two examples, it looks like it's going to be pretty tedious to determine whether prop_1 is an object with sub properties or is just an array. I also noticed in your original post the strings don't have quotes, but in your recently examples they do. Is that inconsistency in the datafile or just your examples?

ShadowTrolllOP•2y ago

If it has = anywhere in it, it is sub property My bad, string have quotes No wait Okay its cancer

ShadowTrolllOP•2y ago

ShadowTrolllOP•2y ago

I am okay with treating everything as strings tho It is just a hobby project of mine and I am using this to mostly display and filter stuff And yeah, I also have the urge to strangle the person who decided booleans are yes/no instead of true/false

Vi Ness•2y ago

Ah, it looks like category would be an enum and is_rare is a bool so they prob'ly don't store them directly as strings Right?!

ShadowTrolllOP•2y ago

Yeeeah true, since AFAIK there are only few categories Also, whatever way I decide to go, there are quite a few properties an object can have (name, cost, area, category, prereq, weight, weight_mod. is_rare. potential and probably few more), how would you recommend to store them? I think objects for sure but dunno if each one should get its own property or just like... one "dictionary" of properties per object

Vi Ness•2y ago

Well, you'd prob'ly make your own objects that represent the objects in the game. Like if the example you posted is some kind of Unit, a Unit could have a tier, category, prerequisites, is_rare, etc. But there might be something like a Building or Upgrade that need other things

ShadowTrolllOP•2y ago

What I am trying to make is an app that allows to search technologies by filtering these properties in some way and shows you the technologies required to obtain it, together with cost and other stuff So (sadly?) this has to all be in one object type

Vi Ness•2y ago

I see, and not every Technology has all of these properties?

ShadowTrolllOP•2y ago

Yeah. Because game and mod devs :/

Vi Ness•2y ago

Are there different kinds of Technologies, like ways you could group them together where each Technology in a group would have the same properties?

ShadowTrolllOP•2y ago

Nope, random bullshit go I tried some string splitting stuff and it seems I can get away with basically this: each "=" means something is a property each "{" means something is a composite (array/sub(sub,sub,sub)property) Now "just" making it work in code Can I have a list of references to lists in C#?

Vi Ness•2y ago

Like a List<List<string>>? Yeah

ShadowTrolllOP•2y ago

I could make a list represent a queue of "how deep" i am putting stuff (appends with "{", pops with "}") and just say whatever property = value I have I put in the thing that is referenced by top of the queue

Vi Ness•2y ago

Sounds like a good use-case for some good old fashion recursion

ShadowTrolllOP•2y ago

aka if I have this:

tech_swag = {
  cost = 2500
  area = engineering
  weight_modifier = {
    factor = 0.5
    modifier = {
      factor = 0.25
      NOR = {
        has_trait_in_council = { TRAIT = leader_trait_expertise_voidcraft }
    has_trait_in_council = { TRAIT = leader_trait_curator }
      }
    }
    modifier = {
      factor = 1.5
      is_specialist_subject_type = { TYPE = bulwark }
    }
}

tech_swag = {
  cost = 2500
  area = engineering
  weight_modifier = {
    factor = 0.5
    modifier = {
      factor = 0.25
      NOR = {
        has_trait_in_council = { TRAIT = leader_trait_expertise_voidcraft }
    has_trait_in_council = { TRAIT = leader_trait_curator }
      }
    }
    modifier = {
      factor = 1.5
      is_specialist_subject_type = { TYPE = bulwark }
    }
}

I would first create tech with name "tech_swag" and create a list, pointer to which I would put in my queue Now I would find cost = 2500, so I would append 2500 to that list Next is appending area = engineering next is appending NEW LIST with key "weight_modifier" AND putting a pointer to it to the top of queue, so next when I see factor = 0.5 I put that into that new list... Makes sense? BTW yes this is mostly a valid tech

Vi Ness•2y ago

Tfw weight_modifier has two modifiers NotLikeYellow

NotLikeYellow

Pacyfist•2y ago

othert than missing quotes around string it's pretty much JSON. you could be able to use the deserializer, they are pretty smart.

ShadowTrolllOP•2y ago

The full tech looks like this if it helps:

tech_space_defense_station_improvement = {
    cost = @tier3cost3
    area = engineering
    tier = 3
    category = { voidcraft }
    prerequisites = { "tech_starbase_4" "tech_modular_engineering" }
    is_rare = yes
    weight = @tier3weight3

    modifier = {
        starbase_upgrade_cost_mult = -0.15
        starbase_upgrade_speed_mult = 0.25
    }

    weight_modifier = {
        factor = 0.5
        modifier = {
            factor = 0.25
            NOR = {
                has_trait_in_council = { TRAIT = leader_trait_expertise_voidcraft }
                has_trait_in_council = { TRAIT = leader_trait_curator }
            }
        }
        modifier = {
            factor = 1.5
            is_specialist_subject_type = { TYPE = bulwark }
        }
        modifier = {
            factor = @ap_technological_ascendancy_rare_tech
            has_ascension_perk = ap_technological_ascendancy
        }
        modifier = {
            factor = @federation_perk_factor
            has_federation = yes
            federation = {
                has_federation_perk = rare_tech_boost
                any_member = { has_technology = tech_space_defense_station_improvement }
            }
        }
        modifier = {
            factor = 1.25
            OR = {
                has_ethic = ethic_militarist
                has_ethic = ethic_fanatic_militarist
            }
        }
    }

    ai_weight = {

    }
}

tech_space_defense_station_improvement = {
    cost = @tier3cost3
    area = engineering
    tier = 3
    category = { voidcraft }
    prerequisites = { "tech_starbase_4" "tech_modular_engineering" }
    is_rare = yes
    weight = @tier3weight3

    modifier = {
        starbase_upgrade_cost_mult = -0.15
        starbase_upgrade_speed_mult = 0.25
    }

    weight_modifier = {
        factor = 0.5
        modifier = {
            factor = 0.25
            NOR = {
                has_trait_in_council = { TRAIT = leader_trait_expertise_voidcraft }
                has_trait_in_council = { TRAIT = leader_trait_curator }
            }
        }
        modifier = {
            factor = 1.5
            is_specialist_subject_type = { TYPE = bulwark }
        }
        modifier = {
            factor = @ap_technological_ascendancy_rare_tech
            has_ascension_perk = ap_technological_ascendancy
        }
        modifier = {
            factor = @federation_perk_factor
            has_federation = yes
            federation = {
                has_federation_perk = rare_tech_boost
                any_member = { has_technology = tech_space_defense_station_improvement }
            }
        }
        modifier = {
            factor = 1.25
            OR = {
                has_ethic = ethic_militarist
                has_ethic = ethic_fanatic_militarist
            }
        }
    }

    ai_weight = {

    }
}

To be frank I am fine with just using the whole thing in weight_modifier as one string I mostly care about the area, category, prerequisites and somehow showing the weight mods

Vi Ness•2y ago

I'd say start with that then. Just get all of the root-level names and values in some strings and see if you can print them out in pairs. Worry about diving down the layers after

ShadowTrolllOP•2y ago

Allright. For the storing, is

Dictionary<string, VariantType>

Dictionary<string, VariantType>

Viable thing to use if I want string-keyed items and they can be anything?

Vi Ness•2y ago

I've never used VariantType but if it works like that, then it should work for now Missing quotes, commas, and square brackets make it a big task to convert this format into JSON

ShadowTrolllOP•2y ago

I am a genius

ShadowTrolllOP•2y ago

if the 2nd group is { I just go into recursion with whatever follows hope it works, will get back if I have questions Thanks for help so far btw

Pacyfist•2y ago

Chat GPT is awesome for getting DTOs from JSON 😛

message.txt

Pacyfist•2y ago

Not related to the question, just noticed it

ShadowTrolllOP•2y ago

yeah, very cool

Vi Ness•2y ago

Now get ChatGPT to convert the text they have into that DTO

Pacyfist•2y ago

using Newtonsoft.Json;
using System;
using System.IO;

public class Program
{
    public static void Main()
    {
        string fileContent = File.ReadAllText("path_to_your_file.txt");

        try
        {
            TechSpaceDefenseStationImprovementDTO deserializedObj = JsonConvert.DeserializeObject<TechSpaceDefenseStationImprovementDTO>(fileContent);
            
            // Use

using Newtonsoft.Json;
using System;
using System.IO;

public class Program
{
    public static void Main()
    {
        string fileContent = File.ReadAllText("path_to_your_file.txt");

        try
        {
            TechSpaceDefenseStationImprovementDTO deserializedObj = JsonConvert.DeserializeObject<TechSpaceDefenseStationImprovementDTO>(fileContent);
            
            // Use

I'm quite sure it's incorrect

Vi Ness•2y ago

Well JsonConvert is going to expect a JSON format

Pacyfist•2y ago

it is a json format with delimiters being empty strings

ShadowTrolllOP•2y ago

@vi.ness Is there a quick way to test a class function? I dont want other stuff to happen, just create an instance of this and call DeSerialiseTech

using Microsoft.VisualBasic;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Text.RegularExpressions;
using System.Threading.Tasks;

namespace StellTechViewer
{
    internal class TechObj
    {
        public string Name { get; set; }
        public Dictionary<string, VariantType> Properties { get; set; }

        public void DeSerialiseTech(string inp)
        {
            this.Properties = new Dictionary<string, VariantType>();
            Regex rx = new Regex("(\\S+)\\s*=\\s*(\\S*)", RegexOptions.Compiled);
            DeSerialisePart(this.Properties, inp, rx);
        }

        private void DeSerialisePart(Dictionary<string, VariantType> target, string inp, Regex rx)
        {
            foreach (Match ItemMatch in rx.Matches(inp))
            {
                Console.WriteLine(ItemMatch);
            }
        }
    }
}

using Microsoft.VisualBasic;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Text.RegularExpressions;
using System.Threading.Tasks;

namespace StellTechViewer
{
    internal class TechObj
    {
        public string Name { get; set; }
        public Dictionary<string, VariantType> Properties { get; set; }

        public void DeSerialiseTech(string inp)
        {
            this.Properties = new Dictionary<string, VariantType>();
            Regex rx = new Regex("(\\S+)\\s*=\\s*(\\S*)", RegexOptions.Compiled);
            DeSerialisePart(this.Properties, inp, rx);
        }

        private void DeSerialisePart(Dictionary<string, VariantType> target, string inp, Regex rx)
        {
            foreach (Match ItemMatch in rx.Matches(inp))
            {
                Console.WriteLine(ItemMatch);
            }
        }
    }
}

In WPF

Vi Ness•2y ago

For testing stuff, I would just break it out into a console app project. That way you don't have to mess with UI elements You can copy/paste it back in when it works

ShadowTrolllOP•2y ago

Yeeeah I guess, thanks

Pacyfist•2y ago

I asked ChatGPT to write a regex to convert that file into a proper JSON file 😄 I wonder if it works.

message.txt

Vi Ness•2y ago

It certainly tried XD

Pacyfist•2y ago

add using System.Text.RegularExpressions; and it should work

Vi Ness•2y ago

It is working. You can see the is_rare turned to true and a few properties got quotes

Pacyfist•2y ago

well... execute 😛 not "work" Ah right Ok I tried nothing and I'm all out of ideas 🙂

Vi Ness•2y ago

When I told ChatGPT that it wasn't a JSON it tried the split lines approach but that completely breaks on all the modifiers with multiple lines D: I think Shadow will have more luck with the regex they're currently playing with

Pacyfist•2y ago

tell him " You are an old english gentleman and I'm your butler. You want to show me how to deserialize that file into DTOs using C#" at least the reply is somewhat amusing 😛

Pacyfist•2y ago

I replied "this string is not JSON sir" and I got this 😄

message.txt

Vi Ness•2y ago

That's about the same thing I got

Vi Ness•2y ago

With enough poking and prodding, ChatGPT got this far at least. It's really bad about digging into those nested modifiers though

ShadowTrolllOP•2y ago

I mean, thats further than I got so far (:

Vi Ness•2y ago

It's still trying to parse one line at a time but if you wanna reference some of the regex it's using, here's the code https://paste.mod.gg/uzpeleqqpftu/0

BlazeBin - uzpeleqqpftu

A tool for sharing your source code with the world!

ShadowTrolllOP•2y ago

@vi.ness Holy shit I got it. It was painful, looks kinda ugly but it seems to work!

    public string Name { get; set; }
    public List<KeyValuePair<string, Object>> Properties { get; set; }

    public void DeSerialiseTech(string inp)
    {
        this.Properties = new List<KeyValuePair<string, Object>>();
        Regex rx = new Regex("(\\S+)\\s*=\\s*(\\S*)", RegexOptions.Compiled);
        Match rxMatch = rx.Match(inp);
        if (rxMatch.Success)
        {
            this.Name = rxMatch.Groups[1].Value;
            Object properties = DeSerialisePart(inp.Substring(rxMatch.Index + rxMatch.Length), rx);
            try {
                this.Properties = (List<KeyValuePair<string, Object>>)properties;
            } catch { }
        }
    }

    private Object DeSerialisePart(string inp, Regex rx)
    {
        MatchCollection rxMatches = rx.Matches(inp);
        if (rxMatches.Count == 0) return inp;

        List<KeyValuePair<string, Object>> subDict = new List<KeyValuePair<string, Object>>();
        int closeIdx = 0;

        foreach (Match rxMatch in rxMatches)
        {
            if (rxMatch.Index >= closeIdx)
            {
                if (rxMatch.Groups[2].Value == "{")
                {
                    int startIdx = rxMatch.Index;
                    closeIdx = startIdx;
                    int openCnt = 1;
                    foreach (char c in inp.Substring(startIdx + rxMatch.Length))
                    {
                        closeIdx += 1;
                        if (c == '{') openCnt += 1;
                        else if (c == '}') openCnt -= 1;
                        if (openCnt == 0) break;
                    }

                    subDict.Add(new KeyValuePair<string, Object>(rxMatch.Groups[1].Value, DeSerialisePart(inp.Substring(startIdx + rxMatch.Length, closeIdx - startIdx - 1), rx)));
                }
                else subDict.Add(new KeyValuePair<string, Object>(rxMatch.Groups[1].Value, rxMatch.Groups[2].Value));
            }
        }

        return subDict;
    }
}

    public string Name { get; set; }
    public List<KeyValuePair<string, Object>> Properties { get; set; }

    public void DeSerialiseTech(string inp)
    {
        this.Properties = new List<KeyValuePair<string, Object>>();
        Regex rx = new Regex("(\\S+)\\s*=\\s*(\\S*)", RegexOptions.Compiled);
        Match rxMatch = rx.Match(inp);
        if (rxMatch.Success)
        {
            this.Name = rxMatch.Groups[1].Value;
            Object properties = DeSerialisePart(inp.Substring(rxMatch.Index + rxMatch.Length), rx);
            try {
                this.Properties = (List<KeyValuePair<string, Object>>)properties;
            } catch { }
        }
    }

    private Object DeSerialisePart(string inp, Regex rx)
    {
        MatchCollection rxMatches = rx.Matches(inp);
        if (rxMatches.Count == 0) return inp;

        List<KeyValuePair<string, Object>> subDict = new List<KeyValuePair<string, Object>>();
        int closeIdx = 0;

        foreach (Match rxMatch in rxMatches)
        {
            if (rxMatch.Index >= closeIdx)
            {
                if (rxMatch.Groups[2].Value == "{")
                {
                    int startIdx = rxMatch.Index;
                    closeIdx = startIdx;
                    int openCnt = 1;
                    foreach (char c in inp.Substring(startIdx + rxMatch.Length))
                    {
                        closeIdx += 1;
                        if (c == '{') openCnt += 1;
                        else if (c == '}') openCnt -= 1;
                        if (openCnt == 0) break;
                    }

                    subDict.Add(new KeyValuePair<string, Object>(rxMatch.Groups[1].Value, DeSerialisePart(inp.Substring(startIdx + rxMatch.Length, closeIdx - startIdx - 1), rx)));
                }
                else subDict.Add(new KeyValuePair<string, Object>(rxMatch.Groups[1].Value, rxMatch.Groups[2].Value));
            }
        }

        return subDict;
    }
}

Vi Ness•2y ago

Ugly doesn't matter on the rough draft! Let's see some console output with those Properties~

ShadowTrolllOP•2y ago

I dont have console, I did testing via button in my WPF 😄

Vi Ness•2y ago

Well throw some labels up XD

ShadowTrolllOP•2y ago

I looked at the object by hovering

ShadowTrolllOP•2y ago

Vi Ness•2y ago

That's great! Breaking them down from there shouldn't be as bad

ShadowTrolllOP•2y ago

Sure hope so x_x Well, next comes getting this stuff, somehow, over to SQLite database Anyway, thanks again, Good night

Accord•2y ago

Was this issue resolved? If so, run /close - otherwise I will mark this as stale and this post will be archived until there is new activity.

We are a programming server aimed at coders discussing everything related to C# (CSharp) and .NET.

54KMembers

View on Discord

Want results from more Discord servers?

Add your server