Most efficient way to modify json in C#
I am storing data as json string, and before store I need to update a value I know I could deserialize to an interface set the property and serialize again but this would cost a lot at least i think (advantage to ensure it has the key)
I have done with now like this:
Is there any better way to do?
19 Replies
Hi, welcome. I have a few points / questions:
1. Is this a noticeable performance issue / bottleneck? i.e. Does it need to be "efficient"?
2. You might find it more efficient to go direct via the database in some circumstances. SQL Server supports
json_modify T-SQL commands, for instance.
3. Is that NewtonSoft.Json rather than System.Text.Json? You could compare the performance of this approach to JsonNode in System.Text.Json ( https://learn.microsoft.com/en-us/dotnet/standard/serialization/system-text-json/use-dom ).How to use a JSON DOM in System.Text.Json - .NET
Learn how to use a JSON document object model (DOM).
Unknown User•2w ago
Message Not Public
Sign In & Join Server To View
I did not want to go deeper in that, but I am using firebase realtime database that one has a Func which
public Func<Type, string, IDictionary<string, OfflineEntry>> OfflineDatabaseFactory { get; set; }
When user opens the app first time it pulls all data from online it stores in local db (by default it stores in memory but i want to reach the data even if next time the app is opened offline).
So I created a LiteDbOfflineDictionary : IDictionary<string, OfflineEntry> with this when first time all data pulled it stores offline and if any data is changed automatically synces to online.If i was in your place i would try for the lulz to convert the string to a ReadOnlySpan<string> and parse the key you are interested only (to avoid parsing the whole document). This being said i know this solution is hacky but it might be faster than converting the whole json to a JObject. But also i agree that having such logic in a setter is not a great idea.
So why can’t you deserialize it?
OfflineEntry is a given class, which keeps the state of the syncable Data. This class has Key and Data properties both as string. In the example by default key is not part of the Data and when I call realtime.Get method I only get back all the Data as IEnumerable<T>. So if I want to delete one of them unfortunatelly IEnumerable<T> T Data does not contain the Key itself.
So therefore my idea was when initially filling this IDictionary up I set the key also as a property of the Data. So later when I do crud operation in my repository I can easily.
So to set Key as part of the Data I wanted
- check if Data is valid json (it triggers with null if delete operation)
- check if Data has key property
- set key property as the outer key
- store in local lite db
I don't fully understand your use-case, but shouldn't you know the key for any data you're accessing? Storing keys inside the value in a KV store always makes me feel like something isn't quite right. It just provides an opportunity for the "inner" and "outer" keys to get out of sync, especially if you have to jump through this kind of hoop to set it.
Rather than trying to recreate the online store offline, I'd take a step back and try to work out what data you're trying to preserve, and create your offline storage and serialisation against that requirement, rather than trying to serialize and store an already serialized format.
I'd also note that if you are allowing start-up-offline, then you'll also need to have a strategy for re-synchronising changes when getting back online, which can be a harder problem to tackle.
So all the sync solved by FirebaseRealtimeDatabase, it has an Func OfflineDatabaseFactory which is by default in memory. In order to swich to real local db I need to implement what it requires.
you can see img 1.
So therefore I created my LiteDbOfflineDictinory img 2.
So on the img 1 when I call GetItemAsync I get all the lets say TODO object which is cool, my problem If i want to select one from it and delete the delete method requires the key. But in the list of TODO object has no the key at all. That was the reason when initially filling it up lets add the key as well to the TODO so when I get all of them it will have the key and i can delete.
Delete and Get method
Of course I am open for any improvements already thank you guys 🍻
I would add this works already, I can do offline crud and close the app, open again and turn the internet on it sync automatically, also if I change something in the online db.
Unknown User•2w ago
Message Not Public
Sign In & Join Server To View
Thank you for the suggestion, always was confused async or not async in this case.
Unknown User•2w ago
Message Not Public
Sign In & Join Server To View
TLDR on
async/await:
* every .net API that is suffixed with Async (eg: .Read() and .ReadAsync()) => use the Async version
* if the API name ends with Async => await it
* if the API returns a Task => await it
* if you have to await in a method:
* that method must have the async keyword (you could even suffix your function name with Async, up to you)
* if it was returning T (eg:string) => it should now return Task<T> (eg: Task<string>)
* APIs to ban and associated fix:
Unknown User•2w ago
Message Not Public
Sign In & Join Server To View
There are many good uses for getters/setters, but this one is dubious, since it's overloaded with logic, and it has some DAL code, along with JSON parsing. I'd make this bit more imperative i.e. move it to an orchestrating service and inject that where needed.
Also, as people have mentioned before, you should always do async so that you won't run out of threads once 10+ users use your service simultaneously
I see, I will take care of the async await case. Thanks for the input! For the getter and setter situation It is a mobile app so no other user will use simultaneously.
For the OfflineDatabaseFactory as you can see it is getter and setter at the same time
Originally I wanted to solve with method as
public IDictionary<string, OfflineEntry> GetSetOfflineEntries(Type type, string path) but i could not distinguish if the current call wants to set or get, but with the getter and setter way I managed to make it work.
Unknown User•2w ago
Message Not Public
Sign In & Join Server To View
Probably you missunderstand I am not calling network call from getter and setter.
I am using firebase realtime database
It has this FirebaseOption to be able to configure how to deal when there is no internet. So when i save an item I call
string key = _realtimeDb.Post(item); and automatically whats to cach to locally as well.
By default it used in memory IDictinory but with that if i restart the app obviously it clears up, so therefore I decided to use LiteDb which is file based offline db for .net.
So in my case when the user logs in first time it pulls all the data and save offline into lite db. later if i do crud even in online/offline mode via _realtimeDb.Post(item); if there is internet automatically sycns there if no it stores only offline and once it goes online again it syncs the delta from offline -> online.
I agree it should not be set/get when i am reaching Db but as you can see it is predefined for get and set but i do not know how could my setter/gett be replaced if the func requires that. So therefore I implement LiteDb similar to IDic. Of course I am open for improvement for not using getter and setting in case, but no idea.
public OfflineEntry this[string key]
{
get => _db.GetCollection<OfflineEntry>(_collectionName).FindById(key);
set => _db.GetCollection<OfflineEntry>(_collectionName).Upsert(key, value);
}