C#•3y ago

Can't get items to be added to update

I can't seem to get the "update" List filled.. I have this Json data https://hastebin.com/bazadowese.json What I am trying to do, is everything that is not in the targetJson should be added to the "inject" list, and the items that are in the targetJson should be checked against the items in the sourceJson, if they are not in there, done. If they are check if they have differences, if they have differences add that item to the "update" list. Currently it just adds everything from the sourceJson into the "inject" list and doesn't do squat for the update list. Here's the function:

string requestBody = await new StreamReader(req.Body).ReadToEndAsync();
LookupInputs data = JsonConvert.DeserializeObject<LookupInputs>(requestBody);
if (data == null)
    return new BadRequestObjectResult("There was no data provided, or the data was not formatted properly");

Dictionary<string, List<Dictionary<string, JToken>>> result = new()
{
    { "inject", new() },
    { "update", new() }
};

if (data.TargetJson.Count == 0)
{
    result["inject"] = (from item in data.SourceJson select item).ToList();
}
else
{
    result["inject"] = (from source in data.SourceJson
        from target in data.TargetJson
        where source[data.CommonIdentifier] != target[data.CommonIdentifier] && !source.DictEquals(target)
        select source).ToList();

    result["update"] = (from source in data.SourceJson
        from target in data.TargetJson
        where source[data.CommonIdentifier] == target[data.CommonIdentifier] // && !source.DictEquals(target)
        select target).ToList();
}

return new OkObjectResult(results);

string requestBody = await new StreamReader(req.Body).ReadToEndAsync();
LookupInputs data = JsonConvert.DeserializeObject<LookupInputs>(requestBody);
if (data == null)
    return new BadRequestObjectResult("There was no data provided, or the data was not formatted properly");

Dictionary<string, List<Dictionary<string, JToken>>> result = new()
{
    { "inject", new() },
    { "update", new() }
};

if (data.TargetJson.Count == 0)
{
    result["inject"] = (from item in data.SourceJson select item).ToList();
}
else
{
    result["inject"] = (from source in data.SourceJson
        from target in data.TargetJson
        where source[data.CommonIdentifier] != target[data.CommonIdentifier] && !source.DictEquals(target)
        select source).ToList();

    result["update"] = (from source in data.SourceJson
        from target in data.TargetJson
        where source[data.CommonIdentifier] == target[data.CommonIdentifier] // && !source.DictEquals(target)
        select target).ToList();
}

return new OkObjectResult(results);

this is the current result: https://hastebin.com/buxaroneda.json the result I'm expecting is that the item "BookID": "6" is left out of the "inject" list, and added to the "update" list

39 Replies

becquerel•3y ago

Have you confirmed that data.TargetJson.Count is greater than 0?

malkavOP•3y ago

yes, I've been logging all over the place 😅 the data.TargetJson.Count == 1

becquerel•3y ago

ok - didn't think it would be that simple 🙂 might take me a min to parse your logic...

malkavOP•3y ago

😅 sorry if it's a mess to sort through

becquerel•3y ago

something should be in update if: 1. it is in targetJson 2. is also in sourceJson 3. has differences to the corresponding item in sourceJson is that correct? if so, don't you want that !source.DictEquals comment added back in? just as a tip - there is an explicit join syntax in LINQ you can use to make it a bit cleaner

malkavOP•3y ago

Let me "Try" and help if (data.TargetJson.Count == 0) this part just adds all the items from data.SourceJson into the "inject" list of the results now comes the fun part lol result["inject"] = ... is supposed to iterate through both data.TargetJson and data.SourceJson and return all the items from data.SourceJson that are not in data.TargetJson (though I see that it is returning the item if only one property of the item in the Dictionary is different because they won't equal, but whatever) result["update'] = ... is supposed to return the items from data.TargetJson that have the same CommonIdentifier as in data.SourceJson but have the rest of the properties different (so it needs updating)

becquerel•3y ago

var x = from source in data.SourceJson
        join target in data.TargetJson on // blah
        select ...

var x = from source in data.SourceJson
        join target in data.TargetJson on // blah
        select ...

malkavOP•3y ago

yea I was testing if that resulted in something different 😅 not sure if I want to join, since I want the item to be pushed to a new List in the results variable? but I'm not sure what join would do in this line of code

becquerel•3y ago

return all the items from data.SourceJson that are not in data.TargetJson

would data.SourceJson.Except(data.TargetJson) work here?

malkavOP•3y ago

I have that in my DictEquals method Wait, here's that method:

public static bool DictEquals<TKey, TValue>(this Dictionary<TKey, TValue> source, Dictionary<TKey, TValue> target) 
        => source.Count == target.Count && !source.Except(target).Any();

public static bool DictEquals<TKey, TValue>(this Dictionary<TKey, TValue> source, Dictionary<TKey, TValue> target) 
        => source.Count == target.Count && !source.Except(target).Any();

I tested, and that acutally does the Equals for me quite well so far

becquerel•3y ago

the join is equivalent to your thing of using two froms with a where on the commonidentifier

malkavOP•3y ago

Ah fair enough, so it pretty much would do the same but cleaner?

Unknown User•3y ago

Message Not Public

malkavOP•3y ago

Because the JToken represents whatever data in the JSON object would be the value of the key. The string is the key

Unknown User•3y ago

Message Not Public

malkavOP•3y ago

wait, lemme explain, and you can correct me on that 🤣 the example JSON I sent is not the data I will always get... What I will receive is a JSON array of objects. These objects have keys (string) and values (unknown) so it was either dynamic, JValue, JToken, or whichever other type of dynamic data because the values can be int, string, objects, or whatever data type

Unknown User•3y ago

Message Not Public

malkavOP•3y ago

No, no, the example is uniform but the data I will receive is not

Unknown User•3y ago

Message Not Public

malkavOP•3y ago

the code is the actual code I intend to use, but the data I send is not the actual one. the data I will receive is based on whatever the user needs to do a Lookup Transformation on but in SSIS and not in Informatica What I'm finding so far is that source[data.CommonIdentifier] == target[data.CommonIdentifier] isn't returning a value. While I'm sure that in the example data I sent both have a BookID of "6", so I'm a little confused why it won't return a value, not even in this loop:

foreach (Dictionary<string, JToken> source in data.SourceJson)
{
    log.LogDebug("Iterating through both dictionaries to check if the target even gets through");
    foreach (Dictionary<string, JToken> target in data.TargetJson.Where(target =>
                 source[data.CommonIdentifier] == target[data.CommonIdentifier]))
    {
        log.LogDebug("Target item: {T}, {S}", target[data.CommonIdentifier], source[data.CommonIdentifier]);
    }
}

foreach (Dictionary<string, JToken> source in data.SourceJson)
{
    log.LogDebug("Iterating through both dictionaries to check if the target even gets through");
    foreach (Dictionary<string, JToken> target in data.TargetJson.Where(target =>
                 source[data.CommonIdentifier] == target[data.CommonIdentifier]))
    {
        log.LogDebug("Target item: {T}, {S}", target[data.CommonIdentifier], source[data.CommonIdentifier]);
    }
}

Unknown User•3y ago

Message Not Public

malkavOP•3y ago

because I had to have some test data don't I 😅

Unknown User•3y ago

Message Not Public

malkavOP•3y ago

how would you go about this then? I get three values from the user source, target, and commonIdentifier source and target are both an Array of objects and commonIdentifier is the key to which the value should match

Unknown User•3y ago

Message Not Public

malkavOP•3y ago

I use Newtonsoft

Unknown User•3y ago

Message Not Public

malkavOP•3y ago

now we're delving much deeper in "I'm doing it wrong" which is fine, I can learn from this, but the point remains. I get an Array of undetermined JSON objects twice, and I have to parse their data and check if the target misses items from the source, or if the target has items that the source needs to update, or the target has the same items as the source, in which case that item can be skipped.

Unknown User•3y ago

Message Not Public

malkavOP•3y ago

Okay, more or less, let's go with that. Point me in the right direction 😅

MODiX•3y ago

tebeco#0205

you would basically loop over property and match Key then value

Quoted by

<@!689473681302224947> from #Can't get items to be added to update (click here)

React with ❌ to remove this embed.

Unknown User•3y ago

Message Not Public

malkavOP•3y ago

wait that last part I don't get.

Unknown User•3y ago

Message Not Public

malkavOP•3y ago

well I was told not to use Dictionary<string, dynamic> hence I used JToken 😅

Unknown User•3y ago

Message Not Public

malkavOP•3y ago

lol fair enough

Unknown User•3y ago

Message Not Public

malkavOP•3y ago

looks like it. My result is shown as well in the initial question, and the result seems to serialize it properly there. problem so far is that it won't fetch either object that have the same common identifier in them from either source or target Ah, well I found the problem. Now to solve the issue 😅 apparently target[data.CommonIdentifier] results in nothing. and I've logged data.CommonIdentifier with the current dataset, which is BookID (should actually be the key) and I've logged data.TargetJson.First().Keys.First() which is also BookID (should be the key) so I'm not sure why I can't get the BookID from the dataset by using this code:

foreach (Dictionary<string, JToken> source in data.SourceJson)
{
    log.LogDebug("Iterating through both dictionaries to check if the target even gets through");
    foreach (Dictionary<string, JToken> target in data.TargetJson)
    {
        if (target[data.CommonIdentifier] != source[data.CommonIdentifier]) continue;
        log.LogDebug("Target item: {T}, source item: {S}", target[data.CommonIdentifier], source[data.CommonIdentifier]);
    }
}

foreach (Dictionary<string, JToken> source in data.SourceJson)
{
    log.LogDebug("Iterating through both dictionaries to check if the target even gets through");
    foreach (Dictionary<string, JToken> target in data.TargetJson)
    {
        if (target[data.CommonIdentifier] != source[data.CommonIdentifier]) continue;
        log.LogDebug("Target item: {T}, source item: {S}", target[data.CommonIdentifier], source[data.CommonIdentifier]);
    }
}

logged this all the way at the top:

log.LogDebug("See if the keys are actually the same as in the json object: {First} - {Identifier}",
            data.TargetJson.First().Keys.First(), data.CommonIdentifier);

log.LogDebug("See if the keys are actually the same as in the json object: {First} - {Identifier}",
            data.TargetJson.First().Keys.First(), data.CommonIdentifier);

and the result in the logger is this:

 See if the keys are actually the same as in the json object: BookID - BookID

 See if the keys are actually the same as in the json object: BookID - BookID

but this is what I get

Logging even more to check if it can even find the item I am looking for:  << should be 6

Logging even more to check if it can even find the item I am looking for:  << should be 6

when I log this part:

log.LogDebug("Logging even more to check if it can even find the item I am looking for: {S} << should be 6",
            data.TargetJson.First()[data.CommonIdentifier]);

log.LogDebug("Logging even more to check if it can even find the item I am looking for: {S} << should be 6",
            data.TargetJson.First()[data.CommonIdentifier]);

Gaming

Programming

Can't get items to be added to update

Did you find this page helpful?