How to use the same HttpClient instance with different session cookies / proxies?
Hello I am managing thousands of accounts at once (nothing malicious, just for pentesting reasons). And was wondering how I could accomplish this.
For each of these accounts I am currently using one HttpClient. But after a while I get "Unable to read data from the transport connection: Connection reset by peer". I believe this is due to Socket exhaustion. And Socket exhaustion is due to the fact that I use so many clients.
18 Replies
Each account has different cookies (authorization etc.) and uses a different proxy. So how can I implement this code pattern by using one single instance for all accounts
You can probably interlace them instead of immediately firing all of them. Knowing your upper limit. I don’t think you’ll do your pen testing any wrong. You could choose to elect more computers, nics, or a local vpn.
the network shouldnt limit me though
like the server has 1gbps and im not sending more than 5mb ps
Speed is irrelevant
Whats relevant?
Amount of socket connections
isnt one socket = one httpclient ?
Mhm
Opened on connect
i dont think i even get close to the max socket amount that is allowed
with 3k httpclients
Just to be sure, how do you instantiate your Http Clients?
MyHttpClient is
public class MyHttpClient : HttpClient
@Joschi
WymJust newing up http clients is not the recommended best practice.
Mainly because it can lead to socket exhaustion. Take a look at the IHttpClientFactory
https://learn.microsoft.com/en-us/dotnet/architecture/microservices/implement-resilient-applications/use-httpclientfactory-to-implement-resilient-http-requests
Use IHttpClientFactory to implement resilient HTTP requests - .NET
Learn how to use IHttpClientFactory, available since .NET Core 2.1, for creating
HttpClient instances, making it easy for you to use it in your applications.Ye thought the same basically but i dont even have that many open
sockets
a fewk
k
the weird thing is after this happens
it works again in 10min
even if I restart right after it doesnt work, i need to wait like 10mins
and its 100% unrelated to the target machine that im requesting
because on my local it doesnt happen
and when I set the proxy locally on my linux server I get:
https://media.discordapp.net/attachments/1089178653486764094/1244058915709391049/image.png?ex=6653bba7&is=66526a27&hm=19ddea3a7f7f3e855b5a319d40113ed0f414724b6455030477c59f883c2582c1&=&format=webp&quality=lossless&width=783&height=408
but on my local machine it works
Connection reset is a funny way of saying disconnect
Current progress:
- Once this happens the server needs 10-15min before I can request it again (a restart wont even fix this)
- I tried to login manually into the proxy and got "Connection reset" in the firefox browser. Tried the same on my local machine and it worked
- I believe it must be the server provider. I think that they have a mechanism that prevents us to send thousands of requests to the same host (the host is the proxy provider)
I think that their mechanism thinks that we are trying to ddos some service or idk
makes literally no sense