C#•6mo ago

✅ EFCore fetching all data in nearly all tables after a single insert

So today I encountered the weirdest error, for which I cannot create a minimal reproducible project. But maybe any of you has any clue on how this is even possible or even why it occours. The project is a .NET7 WebAPI with EFCore pointing towards an SQLServer 2019. We had an object using an auto incrementing int ID.

class MyDbObject
{
  public int Id {get;}
 //...
}

class MyDbObject
{
  public int Id {get;}
 //...
}

We changed that to use a StronglyTypedId

[StronglyTypedId]
readonly partial struct ObjectId
{
  // This and a lot of plumbing and parsong code is source generated, like a EFCore Value converter and so on.
  public int Value {get;}
}

[StronglyTypedId]
readonly partial struct ObjectId
{
  // This and a lot of plumbing and parsong code is source generated, like a EFCore Value converter and so on.
  public int Value {get;}
}

We used this strong ID implementation multiple times and it works without any problems. Now we had a implicit conversion on the ID for convenience.

[StronglyTypedId]
readonly partial struct ObjectId
{
  public static implicit operator ObjectId(MyDbObject obj) => obj.Id;
}

[StronglyTypedId]
readonly partial struct ObjectId
{
  public static implicit operator ObjectId(MyDbObject obj) => obj.Id;
}

Now EFCore started to literally fetch the entire database into memory after doing the single and correct INSERT Always selecting all properties of the entire table and sending it back to the client.

_dbContext.Add(myObjectInstance);
await _dbContext.SaveChangesAsync();

_dbContext.Add(myObjectInstance);
await _dbContext.SaveChangesAsync();

Removing the implicit conversion completely resolved this. This did not happen if it was isolated in a unit test and ran against a test database instance. Because of that it most likely is some configuration on the DBContext or the project. I'm at a complete loss, so maybe one of you has any inkling what the cause of this could be.

6 Replies

tera•6mo ago

how do you know what you described is actually happening? what did you observe?

JoschiOP•6mo ago

We log the EFCore SQL commands in development. The observation being the correct INSERT command being generated and send to the DB first, followed by a series of SELECT [x].Property... (all the properties of the given table) FROM [DB].[TABLE] AS [x] statements without any parameters. We know it is really send back to the client, because we discovered some malformed data in our tables thanks to some EFCore Converter crashing in the process of converting the database value. And also the RAM constantly going up. However I'm not sure if it really sends back the data for all the different tables or just the complete table for the original MyDbObject.

tera•6mo ago

this sounds very odd if what you said were the only changes is project public? bigger picture might help - is it always happening - is it only one place - what else goes on in the method where it happens ... i have no experience with StronglyTypedId but I don't see how just that would cause such behaviour

JoschiOP•6mo ago

No the project is sadly not public. - It is always happening. - I hope I find some time to test this. - It takes in a DTO, converts it to the db model, saves it to the db and returns the newly saved object back to the controller, but we don't reach that point. My best guess would be something in some middleware. But I have no idea what this "something" could be.

tera•6mo ago

do you use lazy loading can you at least show full method code? do you set up anything in the dbcontext? at least i would try strip down the method to minimal repro.. then share sharing just "important bits" might miss something so when i ask what you do, i generally expect code :p

JoschiOP•6mo ago

I know it's sparse, I don't have access to discord and the code on the same machine We found the problem today. As one may expect it was not directly related to the implicit operator. That was just a fluke / not fully rebuilding as it seems. The problem occurred, when the operation threw a DBUpdateException. Which then lead Serilog.Exceptions to reflect and deserialize the complete DBContext. Which of course itterated over all datasets, fetching the entire database. But all this happened, before it even logged that an UpdateError occurred or the exception bubbled to the top, which made it hard to find. That's a known problem and Serilog.Exceptions.EfCore provides a destructerer, which seemingly needs to be set explicitly in code.

Gaming

Programming

✅ EFCore fetching all data in nearly all tables after a single insert