no >> body
no >> body
PostsComments
CC#
Created by no >> body on 7/18/2024 in #help
✅ EF core transactions in PostgreSQL
Yeah, this is exactly what I thought about. If I don't need to read something before updating, then I don't need a transaction either. I also have a place in my code where I create/delete something alongside with updating balances. This still requires a transaction in case something fails, but I reckon I don't need it to be Serializable now
17 replies
CC#
Created by no >> body on 7/18/2024 in #help
✅ EF core transactions in PostgreSQL
This not only fixed my problem but improved performance as well. Thanks!
17 replies
CC#
Created by no >> body on 7/18/2024 in #help
✅ EF core transactions in PostgreSQL
Hmmm, something new. Haven't seen this method before. Let me check
17 replies
CC#
Created by no >> body on 7/18/2024 in #help
✅ EF core transactions in PostgreSQL
Unfortunately, I haven't found a better way to implement retries with transactions in EF Core with PostgreSQL. I believe I dug my own grave when I chose EF for that project. But it is what it is, the code is running already in production and I have to fix this issue with concurrent transactions. Also, FirstAsync and ReloadAsync didn't solve my problems. I still have issues with balances.
17 replies
CC#
Created by no >> body on 7/18/2024 in #help
✅ EF core transactions in PostgreSQL
Hmm, I changed FindAsync to FirstAsync and it looks like the problem is solved. As far as I know Find caches result, so It could use entity with not actual results. Another approach to solve this was to add await context.Entry(walletBalance).ReloadAsync(); inside transaction body
17 replies
CC#
Created by no >> body on 7/18/2024 in #help
✅ EF core transactions in PostgreSQL
I have a feeling that I need something like SELECT FOR UPDATE here to prevent other from reading, while I'm doing transaction
17 replies
CC#
Created by no >> body on 7/18/2024 in #help
✅ EF core transactions in PostgreSQL
So, the retry policy is pretty aggressive. It has 20 retries with a max delay of 30s, which should be more than enough.
17 replies
CC#
Created by no >> body on 7/18/2024 in #help
✅ EF core transactions in PostgreSQL
where DetachUpdatedEntities is: 
    private static void DetachUpdatedEntities(DbContext context)
    {
        var changedEntries = context.ChangeTracker.Entries()
            .Where(e => e.State is EntityState.Added or EntityState.Modified or EntityState.Deleted)
            .ToList();

        foreach (EntityEntry entry in changedEntries)
        {
            entry.State = EntityState.Detached;
        }
    }
    private static void DetachUpdatedEntities(DbContext context)
    {
        var changedEntries = context.ChangeTracker.Entries()
            .Where(e => e.State is EntityState.Added or EntityState.Modified or EntityState.Deleted)
            .ToList();

        foreach (EntityEntry entry in changedEntries)
        {
            entry.State = EntityState.Detached;
        }
    }
17 replies
CC#
Created by no >> body on 7/18/2024 in #help
✅ EF core transactions in PostgreSQL
    public static async Task ExecuteInTransactionAsync(
        this DbContext context,
        Func<CancellationToken, Task> operation,
        IsolationLevel isolationLevel = IsolationLevel.Serializable,
        int maxRetries = 20,
        CancellationToken ct = default)
    {
        var retryCount = 0;
        while (true)
        {
            IDbContextTransaction? dbTransaction = null;
            try
            {
                dbTransaction = await context.Database.BeginTransactionAsync(isolationLevel, ct);
                await operation(ct);
                await context.SaveChangesAsync(false, ct);
                await dbTransaction.CommitAsync(ct);
                context.ChangeTracker.AcceptAllChanges();
                return;
            }
            catch (Exception ex)
            {
                Exception baseException = ex.GetBaseException();
                if (baseException is PostgresException { SqlState: PostgresErrorCodes.SerializationFailure })
                {
                    if (retryCount++ > maxRetries)
                    {
                        throw;
                    }

                    DetachUpdatedEntities(context);
                }
                else
                {
                    if (dbTransaction is not null)
                    {
                        await dbTransaction.RollbackAsync(ct);
                    }

                    throw;
                }
            }
            finally
            {
                if (dbTransaction is not null)
                {
                    await dbTransaction.DisposeAsync();
                }
            }
        }
    }
    public static async Task ExecuteInTransactionAsync(
        this DbContext context,
        Func<CancellationToken, Task> operation,
        IsolationLevel isolationLevel = IsolationLevel.Serializable,
        int maxRetries = 20,
        CancellationToken ct = default)
    {
        var retryCount = 0;
        while (true)
        {
            IDbContextTransaction? dbTransaction = null;
            try
            {
                dbTransaction = await context.Database.BeginTransactionAsync(isolationLevel, ct);
                await operation(ct);
                await context.SaveChangesAsync(false, ct);
                await dbTransaction.CommitAsync(ct);
                context.ChangeTracker.AcceptAllChanges();
                return;
            }
            catch (Exception ex)
            {
                Exception baseException = ex.GetBaseException();
                if (baseException is PostgresException { SqlState: PostgresErrorCodes.SerializationFailure })
                {
                    if (retryCount++ > maxRetries)
                    {
                        throw;
                    }

                    DetachUpdatedEntities(context);
                }
                else
                {
                    if (dbTransaction is not null)
                    {
                        await dbTransaction.RollbackAsync(ct);
                    }

                    throw;
                }
            }
            finally
            {
                if (dbTransaction is not null)
                {
                    await dbTransaction.DisposeAsync();
                }
            }
        }
    }
17 replies
CC#
Created by no >> body on 7/18/2024 in #help
✅ EF core transactions in PostgreSQL
    public static async Task ExecuteInCustomStrategyAsync(
        this DbContext context,
        Func<CancellationToken, Task> operation,
        int maxRetries = 20,
        int delayMilliseconds = 1,
        CancellationToken ct = default)
    {
        IExecutionStrategy strategy = context.Database.CreateExecutionStrategy();
        await strategy.ExecuteAsync(
            (operation, maxRetries, delayMilliseconds, ct),
            async x => await ExecuteWithRetriesAsync(x.operation, x.maxRetries, x.delayMilliseconds, x.ct));
    }
    public static async Task ExecuteInCustomStrategyAsync(
        this DbContext context,
        Func<CancellationToken, Task> operation,
        int maxRetries = 20,
        int delayMilliseconds = 1,
        CancellationToken ct = default)
    {
        IExecutionStrategy strategy = context.Database.CreateExecutionStrategy();
        await strategy.ExecuteAsync(
            (operation, maxRetries, delayMilliseconds, ct),
            async x => await ExecuteWithRetriesAsync(x.operation, x.maxRetries, x.delayMilliseconds, x.ct));
    }
Where ExecuteWithRetriesAsync is: 
    private static async Task ExecuteWithRetriesAsync(
        Func<CancellationToken, Task> operation,
        int maxRetries,
        int delayMilliseconds,
        CancellationToken ct)
    {
        var retryCount = 0;
        int delay = delayMilliseconds;
        while (true)
        {
            try
            {
                await operation(ct);
                return;
            }
            catch (NpgsqlException npgsqlException) when (npgsqlException.InnerException is EndOfStreamException)
            {
                if (retryCount++ > maxRetries)
                {
                    throw;
                }

                delay = Math.Min((int)Math.Pow(delay, 2) * retryCount, 30000);
                int jitter = Random.Shared.Next(-delay / 4, delay / 4);
                int nextDelay = delay + jitter;
                await Task.Delay(nextDelay, ct);
            }
        }
    }
    private static async Task ExecuteWithRetriesAsync(
        Func<CancellationToken, Task> operation,
        int maxRetries,
        int delayMilliseconds,
        CancellationToken ct)
    {
        var retryCount = 0;
        int delay = delayMilliseconds;
        while (true)
        {
            try
            {
                await operation(ct);
                return;
            }
            catch (NpgsqlException npgsqlException) when (npgsqlException.InnerException is EndOfStreamException)
            {
                if (retryCount++ > maxRetries)
                {
                    throw;
                }

                delay = Math.Min((int)Math.Pow(delay, 2) * retryCount, 30000);
                int jitter = Random.Shared.Next(-delay / 4, delay / 4);
                int nextDelay = delay + jitter;
                await Task.Delay(nextDelay, ct);
            }
        }
    }
17 replies
CC#
Created by no >> body on 7/26/2023 in #help
❔ WebApplicationFactory and integration tests
public static IHostBuilder AddDbContexts<TMaster, TReplica>(
this IHostBuilder hostBuilder,
bool initializeMaster,
bool initializeReplica)
where TMaster : DbContext
where TReplica : DbContext
=> hostBuilder.ConfigureServices(
    (hostContext, services) =>
    {
        DatabaseConfiguration databaseConfiguration =
            hostContext.Configuration.GetSection(DatabaseConfiguration.SectionName).Get<DatabaseConfiguration>() ??
            throw new ArgumentNullException(nameof(databaseConfiguration));
        // Some DbContext registration stuff
    });
public static IHostBuilder AddDbContexts<TMaster, TReplica>(
this IHostBuilder hostBuilder,
bool initializeMaster,
bool initializeReplica)
where TMaster : DbContext
where TReplica : DbContext
=> hostBuilder.ConfigureServices(
    (hostContext, services) =>
    {
        DatabaseConfiguration databaseConfiguration =
            hostContext.Configuration.GetSection(DatabaseConfiguration.SectionName).Get<DatabaseConfiguration>() ??
            throw new ArgumentNullException(nameof(databaseConfiguration));
        // Some DbContext registration stuff
    });
Here's the tricky part: my ConfigureAppConfiguration is called after the Program.cs so at the point when I need the connection strings, they are null. I tried to solve this by moving my configuration setup to IHostBuilder.ConfigureAppConfiguration instead of IWebHostBuilder.ConfigureAppConfiguration, but it didn't work. I feel like I'm missing something basic here. Any advice or pointers would be really appreciated. Thanks!
3 replies
CC#
Created by no >> body on 6/6/2023 in #help
❔ Incremental Generator compilation error
But when I use .netstandard2.1 it fails only when doing dotnet build. The project in IDE looks good, no errors and all generated types are accessible
9 replies
CC#
Created by no >> body on 6/6/2023 in #help
❔ Incremental Generator compilation error
I found the reason already 😅 This is because I used in my source generator project .netstandard 2.1 After changing it to 2.0 everything works perfectly
9 replies
CC#
Created by no >> body on 4/24/2023 in #help
❔ ASP.NET Core authorization server in Kubernetes environment
Why it's unusual? I'm quite new to kubernetes, so probably doing something wrong
18 replies
CC#
Created by no >> body on 4/24/2023 in #help
❔ ASP.NET Core authorization server in Kubernetes environment
Yes, namespace for traefic and cert-manager also includes annotations for Linkerd
18 replies
CC#
Created by no >> body on 4/24/2023 in #help
❔ ASP.NET Core authorization server in Kubernetes environment
I'm using service mesh. So it should be mTLS for all traffic inside node
18 replies
CC#
Created by no >> body on 4/24/2023 in #help
❔ ASP.NET Core authorization server in Kubernetes environment
After adding this everything works correctly 
ApplySelfProxy();
app.UseForwardedHeaders();
app.UseHttpsRedirection();
app.UseStaticFiles();
app.UseRouting();
app.UseAuthentication();
app.UseAuthorization();
app.UseSerilogRequestLogging();
app.MapRazorPages();
app.MapDefaultControllerRoute();
app.Run();

void ApplySelfProxy()
{
    if (!System.Net.NetworkInformation.NetworkInterface.GetIsNetworkAvailable())
    {
        throw new Exception("No network available");
    }

    Logger logger = new LoggerConfiguration().WriteTo.Console().CreateLogger();

    IPHostEntry host = Dns.GetHostEntry(Dns.GetHostName());

    var options = new ForwardedHeadersOptions
    {
        ForwardedHeaders = ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto
    };

    foreach (IPAddress address in host.AddressList)
    {
        options.KnownProxies.Add(address);
        logger.Information("Added proxy: {Address}", address);
    }

    app.UseForwardedHeaders(options);
}
ApplySelfProxy();
app.UseForwardedHeaders();
app.UseHttpsRedirection();
app.UseStaticFiles();
app.UseRouting();
app.UseAuthentication();
app.UseAuthorization();
app.UseSerilogRequestLogging();
app.MapRazorPages();
app.MapDefaultControllerRoute();
app.Run();

void ApplySelfProxy()
{
    if (!System.Net.NetworkInformation.NetworkInterface.GetIsNetworkAvailable())
    {
        throw new Exception("No network available");
    }

    Logger logger = new LoggerConfiguration().WriteTo.Console().CreateLogger();

    IPHostEntry host = Dns.GetHostEntry(Dns.GetHostName());

    var options = new ForwardedHeadersOptions
    {
        ForwardedHeaders = ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto
    };

    foreach (IPAddress address in host.AddressList)
    {
        options.KnownProxies.Add(address);
        logger.Information("Added proxy: {Address}", address);
    }

    app.UseForwardedHeaders(options);
}
18 replies
CC#
Created by no >> body on 4/24/2023 in #help
❔ ASP.NET Core authorization server in Kubernetes environment
And the 10.244.1.22 is an inner ip of the POD where authentification server is running
18 replies
CC#
Created by no >> body on 4/24/2023 in #help
❔ ASP.NET Core authorization server in Kubernetes environment
I found this documentation and have tried to add the same code: https://learn.microsoft.com/en-us/aspnet/core/host-and-deploy/proxy-load-balancer?view=aspnetcore-3.1#other-proxy-server-and-load-balancer-scenarios-1 
builder.Services.Configure<ForwardedHeadersOptions>(
    options =>
    {
        options.ForwardedHeaders = ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto;
    });
...
app.UseForwardedHeaders();
builder.Services.Configure<ForwardedHeadersOptions>(
    options =>
    {
        options.ForwardedHeaders = ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto;
    });
...
app.UseForwardedHeaders();
After that the error changed and now I'm getting this: 
2023-04-24 08:15:18.138 Information - Request starting HTTP/1.1 GET http://auth.domain.com/.well-known/openid-configuration - -
2023-04-24 08:15:18.139 Debug - Wildcard detected, all requests with hosts will be allowed.
2023-04-24 08:15:18.139 Verbose - All hosts are allowed.
2023-04-24 08:15:18.140 Debug - The request is insecure. Skipping HSTS header.
2023-04-24 08:15:18.142 Debug - Unknown proxy: [::ffff:10.244.1.22]:47336
2023-04-24 08:15:18.143 Warning - Failed to determine the https port for redirect.
2023-04-24 08:15:18.144 Debug - The request path /.well-known/openid-configuration does not match a supported file type
2023-04-24 08:15:18.166 Debug - No candidates found for the request path '/.well-known/openid-configuration'
2023-04-24 08:15:18.166 Debug - Request did not match any endpoints
2023-04-24 08:15:18.210 Information - The response was successfully returned as a JSON document: {
  "error": "invalid_request",
  "error_description": "This server only accepts HTTPS requests.",
  "error_uri": "https://documentation.openiddict.com/errors/ID2083"
}.
2023-04-24 08:15:18.138 Information - Request starting HTTP/1.1 GET http://auth.domain.com/.well-known/openid-configuration - -
2023-04-24 08:15:18.139 Debug - Wildcard detected, all requests with hosts will be allowed.
2023-04-24 08:15:18.139 Verbose - All hosts are allowed.
2023-04-24 08:15:18.140 Debug - The request is insecure. Skipping HSTS header.
2023-04-24 08:15:18.142 Debug - Unknown proxy: [::ffff:10.244.1.22]:47336
2023-04-24 08:15:18.143 Warning - Failed to determine the https port for redirect.
2023-04-24 08:15:18.144 Debug - The request path /.well-known/openid-configuration does not match a supported file type
2023-04-24 08:15:18.166 Debug - No candidates found for the request path '/.well-known/openid-configuration'
2023-04-24 08:15:18.166 Debug - Request did not match any endpoints
2023-04-24 08:15:18.210 Information - The response was successfully returned as a JSON document: {
  "error": "invalid_request",
  "error_description": "This server only accepts HTTPS requests.",
  "error_uri": "https://documentation.openiddict.com/errors/ID2083"
}.
18 replies
CC#
Created by no >> body on 4/24/2023 in #help
❔ ASP.NET Core authorization server in Kubernetes environment
So, it looks like even when I call endpoint with https scheme, under the hood it uses http protocol Also, my middleware contains 
app.UseHttpsRedirection();
app.UseHttpsRedirection();
18 replies