C#•9mo ago

Entity Framework - Circular References

Hey everyone. I'm stuck on this issue since yesterday. I'm using Entity Framework (repository & unit of work), AutoMapper and Blazor. I have a endpoint that retrieves, for example, Teams and that allows me to choose which properties to include. The issue is that I am unable prevent circular references from the entities, even when using mapping, for example, Teams references Persons, Person has a Team, so it references Teams, and so on. I have tried MaxDepth, PreserveReferences on mapping, as well as ignoring cycles on JSON and what seems like dozen other suggestions, ironically I think I'm going in circles. This seems to me like it's something that MUST have a easy solution, but I'm not being able to find anything on it. https://paste.mod.gg/nfujgutykvga/0

BlazeBin - nfujgutykvga

A tool for sharing your source code with the world!

17 Replies

Jimmacle•9mo ago

the solution is not to use the same model for EF and JSON serialization EF itself doesn't care about circular references, if your DTOs have circular references that's a different problem and you probably have to manually map to fix the structure i personally don't use any kind of automatic mapper

Pobiega•9mo ago

Use DTOs and either a manual mapping or something like mapperly

pyrodisticOP•9mo ago

I'm not though, right? I'm using, or trying to use, the DTO for the endpoint and the entity for the actual DB request, no? How would I be able, through manual mapping to overcome this issue? Basically all I really want to do is Team -> Person -> Team for this person is already set, and was the origin so I don't map that one.

Jimmacle•9mo ago

it depends exactly what exception is being thrown, i don't have full details of the problem i usually do my mapping by projecting directly into the DTO in the EF query, which is also more efficient because you aren't overquerying for data for example

    public async Task<Result<Response>> Handle(Query request, PortalDbContext db, CancellationToken cancellationToken)
    {
        var details = await db.Projects.AsNoTracking()
            .Where(p => p.Number == request.Number)
            .Select(p => new Response(
                p.Number,
                p.Status,
                p.Title,
                p.Description,
                p.BillingType,
                p.CreatedOn,
                p.StartedOn,
                p.ClosedOn,
                p.FlexProjectNumbers,
                p.LabWork,
                p.QuotePriceDollars,
                p.QuotePriceOther,
                new EmployeeDto(p.Handler.Number, p.Handler.FullName, p.Handler.Email),
                p.Contact!.Id,
                db.Expenses.Where(x => x.ProjectNumber == p.Number && x.BillToCustomer).Sum(x => x.AmountUsd),
                db.Expenses.Where(x => x.ProjectNumber == p.Number).Sum(x => x.AmountUsd)))
            .FirstOrDefaultAsync(cancellationToken);

        return details ?? Result<Response>.Fail("Project not found.");
    }

    public async Task<Result<Response>> Handle(Query request, PortalDbContext db, CancellationToken cancellationToken)
    {
        var details = await db.Projects.AsNoTracking()
            .Where(p => p.Number == request.Number)
            .Select(p => new Response(
                p.Number,
                p.Status,
                p.Title,
                p.Description,
                p.BillingType,
                p.CreatedOn,
                p.StartedOn,
                p.ClosedOn,
                p.FlexProjectNumbers,
                p.LabWork,
                p.QuotePriceDollars,
                p.QuotePriceOther,
                new EmployeeDto(p.Handler.Number, p.Handler.FullName, p.Handler.Email),
                p.Contact!.Id,
                db.Expenses.Where(x => x.ProjectNumber == p.Number && x.BillToCustomer).Sum(x => x.AmountUsd),
                db.Expenses.Where(x => x.ProjectNumber == p.Number).Sum(x => x.AmountUsd)))
            .FirstOrDefaultAsync(cancellationToken);

        return details ?? Result<Response>.Fail("Project not found.");
    }

pyrodisticOP•9mo ago

The exception is currently thrown on: var teamDto = _mapper.Map<TeamDto>(team); AutoMapper.AutoMapperMappingException HResult=0x80131500 Message=Missing type map configuration or unsupported mapping. Source=<Cannot evaluate the exception source> StackTrace: <Cannot evaluate the exception stack trace> If I remove that line and return team I get: System.Text.Json.JsonException: A possible object cycle was detected. This can either be due to a cycle or if the object depth is larger than the maximum allowed depth of 32. Consider using ReferenceHandler.Preserve on JsonSerializerOptions to support cycles. Path: $.Persons.Team.Persons.Team.Persons.Team.Persons.Team.Persons.Team.Persons.Team.Persons.Team.Persons.Team.Persons.Team.Persons.Team.Id. If I add the Preserve to the JSON the DTOs won't work because JSON adds automatic properties to each entry (id, value). And so on.

Jimmacle•9mo ago

yeah so the mapper is the issue i don't know how automapper works, i just know it's not recommended because once the magic breaks you get stuck if you still want a mapping library, iirc mapperly uses source generators so you will get compile time errors for invalid configurations also as an aside, no point in adding unit of work and repositories on top of EF core because the DbContext is a unit of work and the DbSets are repositories

Pobiega•9mo ago

The problem is that your DTOs have the same cyclic relationships that your entities have. Your DTOs should be specificly designed to NOT have that.

Jimmacle•9mo ago

it sounds like the problem is actually that the json related issues appear when the DTOs are skipped entirely because the mapper doesn't work so the mapper configuration being wrong is the real problem the json issues are irrelevant because they're a symptom of trying to serialize EF entities to json which shouldn't be done to begin with

pyrodisticOP•9mo ago

So partially I was being stupid. There's really no error on the mapping. I was missing FirstOrDefault on the Get call. So the only error is only: System.Text.Json.JsonException: A possible object cycle was detected. This can either be due to a cycle or if the object depth is larger than the maximum allowed depth of 32. Consider using ReferenceHandler.Preserve on JsonSerializerOptions to support cycles. Path: $.Persons.Team.Persons.Team.Persons.Team.Persons.Team.Persons.Team.Persons.Team.Persons.Team.Persons.Team.Persons.Team.Persons.Team.Persons. Adding the Json options:

builder.Services.AddControllers().AddJsonOptions(options =>
{
    options.JsonSerializerOptions.Converters.Add(new JsonStringEnumConverter()); //Check if line's needed?
    options.JsonSerializerOptions.ReferenceHandler = ReferenceHandler.Preserve;
});

builder.Services.AddControllers().AddJsonOptions(options =>
{
    options.JsonSerializerOptions.Converters.Add(new JsonStringEnumConverter()); //Check if line's needed?
    options.JsonSerializerOptions.ReferenceHandler = ReferenceHandler.Preserve;
});

The API request for teams and persons results in:

{
  "id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
  "name": "string",
  "shortName": "string",
  "tvName": "string",
  "organisation": "string",
  "gender": "F",
  "historical": true,
  "teamType": "ORG",
  "competitionId": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
  "competition": {
    "id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
    "name": "string",
    "description": "string",
    "startDate": "2024-04-19T21:49:25.752Z",
    "endDate": "2024-04-19T21:49:25.752Z",
    "totalDistance": 0,
    "competitionCode": "string",
    "documentCode": "string"
  },
  "persons": [
    {
      "id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
      "givenName": "string",
      "teamId": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
      "team": "string",
      "competitionId": "3fa85f64-5717-4562-b3fc-2c963f66afa6"
    }
  ]
}

{
  "id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
  "name": "string",
  "shortName": "string",
  "tvName": "string",
  "organisation": "string",
  "gender": "F",
  "historical": true,
  "teamType": "ORG",
  "competitionId": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
  "competition": {
    "id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
    "name": "string",
    "description": "string",
    "startDate": "2024-04-19T21:49:25.752Z",
    "endDate": "2024-04-19T21:49:25.752Z",
    "totalDistance": 0,
    "competitionCode": "string",
    "documentCode": "string"
  },
  "persons": [
    {
      "id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
      "givenName": "string",
      "teamId": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
      "team": "string",
      "competitionId": "3fa85f64-5717-4562-b3fc-2c963f66afa6"
    }
  ]
}

Pobiega•9mo ago

That's just a workaround thou

pyrodisticOP•9mo ago

I guess, because if I have reference I'll have $id, $value and $ref all over and it won't map. I'm actually very lost, I can barely believe that this is something that doesn't have an easy fix. I can't really change the Dtos, I probably will have to forget Dtos and just manually map on each API endpoint. Which is going to be very time consuming.

Pobiega•9mo ago

its not time consuming at all. Takes a few minutes per api endpoint. and yes, tahts how you use DTOs they are associated with their respective endpoints more so than the entites itself

pyrodisticOP•9mo ago

The thing is that this was a simple example to illustrate the issue. I will have dozens of endpoints, and again, I think there must be a simpler way. For example, if I wanted to do get the Teams as it currenly is I would have to do:

var teamDtos = teams.Select(team => new TeamDto
{
    Id = team.Id,
    Name = team.Name,
    // dozens of other properties here...
    CompetitionId = team.CompetitionId,
    Competition = team.Competition == null ? null : new CompetitionDto
    {
        Id = team.Competition.Id,
        Name = team.Competition.Name,
        //dozens of other properties here...
    },
    Persons = team.Persons == null ? null : team.Persons.Select(person => new PersonDto
    {
        Id = person.Id,
        GivenName = person.GivenName,
        //dozens of other properties here...
    }).ToList()
});

var teamDtos = teams.Select(team => new TeamDto
{
    Id = team.Id,
    Name = team.Name,
    // dozens of other properties here...
    CompetitionId = team.CompetitionId,
    Competition = team.Competition == null ? null : new CompetitionDto
    {
        Id = team.Competition.Id,
        Name = team.Competition.Name,
        //dozens of other properties here...
    },
    Persons = team.Persons == null ? null : team.Persons.Select(person => new PersonDto
    {
        Id = person.Id,
        GivenName = person.GivenName,
        //dozens of other properties here...
    }).ToList()
});

All because I the mapping can't ignore the circular reference Team - > Person.Team when detected? I'm going to research a little bit further, because I actually think with some effort I can create a mapping algorithm that will just do that automatically... If the property is a class check if it's the same as a parent class, if yes, skip mapping that class.

Keswiik•9mo ago

Doing what you've shown (to convert the team to a dto) is pretty normal and only takes a couple minutes to set up.

pyrodisticOP•9mo ago

Am I stupid or would it might be more efficient to simple create two Dtos for each entity in a way that circular references wouldn't be an issue? One that references it fully and other to be used when there's an innumerable that references the same property.

public class TeamDto
{
    public Guid? Id { get; set; }
    public string Name { get; set; }
    public Guid CompetitionId { get; set; }
    public CompetitionDto Competition { get; set; }
    public List<PersonBaseDto> Persons { get; set; }
}

public class PersonDto : PersonBaseDto
{
    public TeamDto Team { get; set; }
}

public class PersonSimpleDto
{
    public Guid? Id { get; set; }
    public Guid TeamId { get; set; }
    public Guid CompetitionId { get; set; }
}

CreateMap<PersonDto, Person>()
   .ForMember(dest => dest.Id, opt => opt.Ignore());
CreateMap<Person, PersonDto>();
CreateMap<Person, PersonSimpleDto>();

public class TeamDto
{
    public Guid? Id { get; set; }
    public string Name { get; set; }
    public Guid CompetitionId { get; set; }
    public CompetitionDto Competition { get; set; }
    public List<PersonBaseDto> Persons { get; set; }
}

public class PersonDto : PersonBaseDto
{
    public TeamDto Team { get; set; }
}

public class PersonSimpleDto
{
    public Guid? Id { get; set; }
    public Guid TeamId { get; set; }
    public Guid CompetitionId { get; set; }
}

CreateMap<PersonDto, Person>()
   .ForMember(dest => dest.Id, opt => opt.Ignore());
CreateMap<Person, PersonDto>();
CreateMap<Person, PersonSimpleDto>();

Jimmacle•9mo ago

DTOs should precisely match the intended shape of the data and not rely on magic to fix design issues also, polymorphic serialization is another thing to deal with

pyrodisticOP•9mo ago

So I should just stop trying to dig a deeper hole and simply do this, and add properties as they are needed?

var teamDtos = teams.Select(team => new TeamDto
{
    Id = team.Id,
    Name = team.Name,
    // dozens of other properties here...
    CompetitionId = team.CompetitionId,
    Competition = team.Competition == null ? null : new CompetitionDto
    {
        Id = team.Competition.Id,
        Name = team.Competition.Name,
        //dozens of other properties here...
    },
    Persons = team.Persons == null ? null : team.Persons.Select(person => new PersonDto
    {
        Id = person.Id,
        GivenName = person.GivenName,
        //dozens of other properties here...
       //excludes TeamDto
    }).ToList()
});

var teamDtos = teams.Select(team => new TeamDto
{
    Id = team.Id,
    Name = team.Name,
    // dozens of other properties here...
    CompetitionId = team.CompetitionId,
    Competition = team.Competition == null ? null : new CompetitionDto
    {
        Id = team.Competition.Id,
        Name = team.Competition.Name,
        //dozens of other properties here...
    },
    Persons = team.Persons == null ? null : team.Persons.Select(person => new PersonDto
    {
        Id = person.Id,
        GivenName = person.GivenName,
        //dozens of other properties here...
       //excludes TeamDto
    }).ToList()
});

Gaming

Programming

Entity Framework - Circular References