C#•14mo ago

❔ CookieContainer and IHttpClientFactory

Hello, I'm currently migrating some old code base (.Net Core 3.0), and the code itself has a bunch of code smells, so I'm trying to improve it a bit (a bunch). It was a big Web Service and I'm also splitting it into smaller Web Services. I'm looking to take advantage of the IHtppClientFactory. I understand the concept, it reuses a HttpClient object for the connections, but I have a issue. The old code did something like this in one of the Controllers:

public class myClass
{
  private HttpClient _client;
  private CookieContainer _cookieContainer;
  private HttpClientHandler _httpClientHandler;

  public myClass() 
  {
    _cookieContainer = new CookieContainer();
    _httpClientHandler= new HttpClientHandler
    {
      AllowAutoRedirect = false,
      CookieContainer = _cookieContainer,
      AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate,                
    };

    _client= new HttpClient(_httpClientHandler);
    _client.Timeout = TimeSpan.FromSeconds(30);
    ConfigureDefaultHeaders();
  }
  
  private void ConfigureDefaultHeaders()
  {
    client.DefaultRequestHeaders.Add("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36");
  }
  // ...
}

public class myClass
{
  private HttpClient _client;
  private CookieContainer _cookieContainer;
  private HttpClientHandler _httpClientHandler;

  public myClass() 
  {
    _cookieContainer = new CookieContainer();
    _httpClientHandler= new HttpClientHandler
    {
      AllowAutoRedirect = false,
      CookieContainer = _cookieContainer,
      AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate,                
    };

    _client= new HttpClient(_httpClientHandler);
    _client.Timeout = TimeSpan.FromSeconds(30);
    ConfigureDefaultHeaders();
  }
  
  private void ConfigureDefaultHeaders()
  {
    client.DefaultRequestHeaders.Add("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36");
  }
  // ...
}

21 Replies

júlio•14mo ago

From what I understand, this class is used to make requests to whatever URL I send it. The thing is, the places apparently use cookies for subsequent requests. Cookies that are stored in a CookieContainer. Each Controller would have a instance of this class in them. My question is, can I improve this with Dependency Injection and IHttpClientFactory? I experimented a bit but I wasn't able to figure out how to have a CookieContainer without a HttpClientHandler. Does anyone know better?

Henkypenky•14mo ago

you can use typed clients and configure each client specifically for each service you want for example

builder.Services.AddHttpClient<TClass>(httpClient =>
{
    httpClient.BaseAddress = new Uri("url");
    httpClient.Timeout = TimeSpan.FromSeconds(30);
    //etc. httpclient properties here
}).ConfigurePrimaryHttpMessageHandler(() =>
{
    return new HttpClientHandler
    {
        //here cookies
        UseCookies = true,
        AllowAutoRedirect = false,
        //more properties of HttpClientHandler
    };
});

builder.Services.AddHttpClient<TClass>(httpClient =>
{
    httpClient.BaseAddress = new Uri("url");
    httpClient.Timeout = TimeSpan.FromSeconds(30);
    //etc. httpclient properties here
}).ConfigurePrimaryHttpMessageHandler(() =>
{
    return new HttpClientHandler
    {
        //here cookies
        UseCookies = true,
        AllowAutoRedirect = false,
        //more properties of HttpClientHandler
    };
});

this way you will not share the http message handler that IHttpClientFactory has, because it's pooled, it will be isolated to your typed implementation now in your class just simple DI

public class TClass
{
    private readonly HttpClient _httpClient;

    public TClass(HttpClient httpClient)
    {
        _httpClient = httpClient;
    }

    public async Task<T> DoStuff() 
    {
        //consume
        var response = await _httpclient.GetAsync("url");
        return T;
    }
}

public class TClass
{
    private readonly HttpClient _httpClient;

    public TClass(HttpClient httpClient)
    {
        _httpClient = httpClient;
    }

    public async Task<T> DoStuff() 
    {
        //consume
        var response = await _httpclient.GetAsync("url");
        return T;
    }
}

so you are on the right track, use IHttpClientFactory, but isolate the pooled HttpMessageHandler that other implementations of the factory use to a typed mode. Basically you want all your configuration on the service registration and configuration and the consumption on the service via DI after all this, it's a good idea to implement/override outgoing request middleware with a delegating handler. Basically you want to override SendAsync (which is the base version that is used when you call all of your fancy httpclient methods) so this way you "intercept" the request the moment is about to go out and you can do some cookie management if you want + a bunch of other things give this a read https://learn.microsoft.com/en-us/aspnet/core/fundamentals/http-requests?view=aspnetcore-7.0 one more thing, as your are gonna read in the article, you will have to be really careful with cookies and the factory because everything is shared my recommendation is that you disable cookies alltogether and manage them manually using handlers for example: if you have an api that sets a cookie on a response header, the typed client will use that for subsequent requests, so you lose control, so in my opinion you will have to end up using a handler for all this specifically read the section: outogoing request middleware

júlio•14mo ago

What woudl tghe TClass be? The controller type for that particular HttpClient? Sorry for late reply I have limited opportunities to be in Discord during the week

Henkypenky•14mo ago

likely a controller you can call, or a service that you can then inject

Accord•14mo ago

Was this issue resolved? If so, run /close - otherwise I will mark this as stale and this post will be archived until there is new activity.

júlio•14mo ago

Is it possible to have a IHttpClientFactory that gives me HttpClients with a non-shared CookieContainer?

Henkypenky•14mo ago

nope you can get close with typed clients but no it's not possible

Accord•14mo ago

Was this issue resolved? If so, run /close - otherwise I will mark this as stale and this post will be archived until there is new activity.

júlio•14mo ago

What about with something like this?

services.AddHttpClient("CookieClient")
        .ConfigureHttpClient((serviceProvider, httpClient) =>
        {
            var cookieContainer = new CookieContainer();
            var handler = new HttpClientHandler
            {
                CookieContainer = cookieContainer,
                UseCookies = true,
                UseDefaultCredentials = false,
            };

            httpClient.BaseAddress = new Uri("https://example.com"); // Set the base URL for your target website.
            httpClient.DefaultRequestHeaders.Accept.Clear();
            httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));

            httpClient.Timeout = TimeSpan.FromSeconds(30); // Set the desired timeout value.

            httpClient.DefaultRequestHeaders.Add("User-Agent", "Your User Agent"); // Set the User-Agent header if required.

            httpClient.HttpMessageHandler = handler;
        });

services.AddHttpClient("CookieClient")
        .ConfigureHttpClient((serviceProvider, httpClient) =>
        {
            var cookieContainer = new CookieContainer();
            var handler = new HttpClientHandler
            {
                CookieContainer = cookieContainer,
                UseCookies = true,
                UseDefaultCredentials = false,
            };

            httpClient.BaseAddress = new Uri("https://example.com"); // Set the base URL for your target website.
            httpClient.DefaultRequestHeaders.Accept.Clear();
            httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));

            httpClient.Timeout = TimeSpan.FromSeconds(30); // Set the desired timeout value.

            httpClient.DefaultRequestHeaders.Add("User-Agent", "Your User Agent"); // Set the User-Agent header if required.

            httpClient.HttpMessageHandler = handler;
        });

and the inject a IHttpClientFactory in the class i need to make requests?

Henkypenky•14mo ago

those are named clients, you will be creating a client, using it and disposing for each request you make, i guess it can work, but it will drastically increase your memory and you will lose performance. If neither of these things are a problem go for it @dryfish

júlio•14mo ago

I see, bummer :P I'll keep digging maybe i'll find some way ChatGPT told me to try and use a DelegatingHandler Imma look into that ig It's almost like a transaction The pattern is simple log in, store cookies, do stuff, log out and sipose there must be a way to reuse the mechanism to make the connection (HttpClient), while still managing the rest

Henkypenky•14mo ago

i already suggested you that it is the proper way you mange the cookies and turn of automatic cookies

júlio•14mo ago

I will re read everything then, thank you!

Henkypenky•14mo ago

júlio•14mo ago

main issue it that Well not really a issue more of a burden I have many links and the configuration of the Factory would be very long :P

Henkypenky•14mo ago

that's normal

júlio•14mo ago

i think this cant be done i will always need to create a new httpclient becasue i have no way to handle asynchronous use of the same httpclient instance and since the instance would use the cookiecontainer either its shared wich i dont want or its not shared but that cant really be easly addressed and even if i implemented some sort of mutex mechanism it would slow things down a lot

Henkypenky•14mo ago

you keep coming back to the same thing so final time typed clients delegating handler(s) manage the cookies yourself no cookie container take control of the pipeline

júlio•14mo ago

But that will result in the same issue I would still need a instance of HttpClient per request

Henkypenky•14mo ago

yes, but i fail to see the issue, typed clients are transient they are created, consumed and disposed they also separate concerns and only live in the implementation that is registered so you can manipulate different cookies with different handlers depending on what implementation named clients have a bit more overhead than typed clients

Accord•14mo ago

Was this issue resolved? If so, run /close - otherwise I will mark this as stale and this post will be archived until there is new activity.

Gaming

Programming

❔ CookieContainer and IHttpClientFactory