C#•15mo ago

❔ Multithreading an async function

I have an async function that I want to multithread with n number of threads, but I get convert error if I initialize the thread with new Thread(methodName) From my understanding, I need to use the .Wait() method every time I call the request method, because it has a Task object.

int threadNumber = 5;
Thread[] threads = new Thread[threadNumber];

for (int i = 0; i < threadNumber; i++)
{
  threads[i] = new Thread(request);
  threads[i].Start();
}

for (int i = 0; i < threadNumber; i++) threads[i].Join();

int threadNumber = 5;
Thread[] threads = new Thread[threadNumber];

for (int i = 0; i < threadNumber; i++)
{
  threads[i] = new Thread(request);
  threads[i].Start();
}

for (int i = 0; i < threadNumber; i++) threads[i].Join();

The method request():

static async Task request()
{
  using var client = new HttpClient();
  client.Timeout = TimeSpan.FromSeconds(5);
  var res = await client.GetAsync(URI);
}

static async Task request()
{
  using var client = new HttpClient();
  client.Timeout = TimeSpan.FromSeconds(5);
  var res = await client.GetAsync(URI);
}

73 Replies

Knuceles•15mo ago

This is the error I get from initializing thread with the method

Knuceles•15mo ago

Knuceles•15mo ago

If I change the return type from Task to void, then I can initialize it, but the method doesn't complete at the await key

chef drone builder•15mo ago

don't manually create new threads, use the threadpool with Task.Run or the Parallel class dont create a new httpclient on each request, you should only use one across your application but you shouldn't run this method on a new thread, because it isn't cpu bound work you could do something like this:

var tasks = Enumerable.Range(0, 5).Select(x => request());
await Task.WhenAll(tasks);

var tasks = Enumerable.Range(0, 5).Select(x => request());
await Task.WhenAll(tasks);

Knuceles•15mo ago

But this isn't making threads Doesn't WhenAll wait until all the code is executed, but doesn't it do it synchronously, rather than creating multiple threads

chef drone builder•15mo ago

Why do you want to create multiple threads? If you want to run the requests in parallel, then this works An if you really want multiple threads then just use the methods i mentioned before and NOT new Thread()

Pobiega•15mo ago

your request method should work just fine in a single thread its just waiting for IO thats literally the ideal usecase for async :p

Knuceles•15mo ago

I'm trying to send multiple http requests at once, and some requests would get stuck in a timeout, so I think using multiple thread is necessary unless I'm missing something

x0rld•15mo ago

the new HttpClient() in each request catsweat

Pobiega•15mo ago

absolutely not.

Knuceles•15mo ago

I'm setting a new proxy address for each request. dang it, i forgot that i can just override those address lol

Pobiega•15mo ago

you can make multiple requests at once without having multiple threads. as long as you don't use any blocking code, you're fine

Knuceles•15mo ago

Thank! The Task.Run works perfectly fine with this. but for educational purpose, is there a way to do it with multi threads?

Pobiega•15mo ago

Task.Run will use multiple threads.

Knuceles•15mo ago

Do you have an example?

chef drone builder•15mo ago

don't use Task.Run for this!!!! you don't need multiple threads to send the request at the same time

Pobiega•15mo ago

var client = new HttpClient();
var listOfUrls = new List<string>
{
    "https://www.google.com",
    "https://www.microsoft.com",
    "https://www.bing.com",
};
var tasks = listOfUrls.Select(url => client.GetAsync(url)).ToArray();
var results = await Task.WhenAll(tasks);

var client = new HttpClient();
var listOfUrls = new List<string>
{
    "https://www.google.com",
    "https://www.microsoft.com",
    "https://www.bing.com",
};
var tasks = listOfUrls.Select(url => client.GetAsync(url)).ToArray();
var results = await Task.WhenAll(tasks);

chef drone builder•15mo ago

Knuceles•15mo ago

oh this is fascinating I need to learn this separately thank you guys !close

Accord•15mo ago

Closed!

Knuceles•15mo ago

new problem

static async void Test()
{
  var client = new HttpClient();

  Task[] tasks = Enumerable.Range(0, 5).Select(async x =>
  {
    var res = await client.GetAsync(URL);
    Console.WriteLine(res.StatusCode);
    }).ToArray();
    await Task.WhenAll(tasks);
  }
}

static async void Test()
{
  var client = new HttpClient();

  Task[] tasks = Enumerable.Range(0, 5).Select(async x =>
  {
    var res = await client.GetAsync(URL);
    Console.WriteLine(res.StatusCode);
    }).ToArray();
    await Task.WhenAll(tasks);
  }
}

When I execute Test(), it seems like the code doesn't wait for the await part in the function but if i replace await client.getasync(url) to something that doesnt require async, the code runs

cap5lut•15mo ago

if just call Test(); instead of await Test(); its basically fire and forget

ffmpeg -i me -f null -•15mo ago

await Task.WhenAll doesn't seem correct there it should be outside the lambda

cap5lut•15mo ago

instead of thinking about multi threading u should first think about how async works

Knuceles•15mo ago

🤦 totally forgot

cap5lut•15mo ago

also u should not use async void but async Task

Pobiega•15mo ago

public static async Task Main()
{
    var client = new HttpClient();
    var listOfUrls = new List<string>
    {
        "https://www.google.com",
        "https://www.microsoft.com",
        "https://www.bing.com",
    };
    var tasks = listOfUrls.Select(url => GetStatusCode(client, url)).ToArray();
    var results = await Task.WhenAll(tasks);

    foreach (var statusCode in results)
    {
        Console.WriteLine(statusCode);
    }

}

private static async Task<HttpStatusCode> GetStatusCode(HttpClient client, string url)
{
    var resp = await client.GetAsync(url);
    return resp.StatusCode;
}

public static async Task Main()
{
    var client = new HttpClient();
    var listOfUrls = new List<string>
    {
        "https://www.google.com",
        "https://www.microsoft.com",
        "https://www.bing.com",
    };
    var tasks = listOfUrls.Select(url => GetStatusCode(client, url)).ToArray();
    var results = await Task.WhenAll(tasks);

    foreach (var statusCode in results)
    {
        Console.WriteLine(statusCode);
    }

}

private static async Task<HttpStatusCode> GetStatusCode(HttpClient client, string url)
{
    var resp = await client.GetAsync(url);
    return resp.StatusCode;
}

Florian Voß•15mo ago

just for my understanding, doesn't your solution also use multiple threads? this one

Pobiega•15mo ago

no only one thread there.

cap5lut•15mo ago

arent the tasks executed on the thread pool and thus its simply not defined on which threads its running?

Pobiega•15mo ago

I highly recommend reading $nothread

MODiX•15mo ago

https://blog.stephencleary.com/2013/11/there-is-no-thread.html

There Is No Thread

This is an essential truth of async in its purest form: There is no thread.

Pobiega•15mo ago

the tl;dr is that no, not every task/await results in work being done on the threadpool

cap5lut•15mo ago

basically each task gets scheduled, runs on a non busy thread til its execution is paused, once it can resume it will continue execution on one of the threads again? eg, the requests will be built on one or more threads of the thread pool, etc. then the task execution will be suspended until the request is sent and response comes in, then the task execution continues on a non busy thread again

Knuceles•15mo ago

With this code, how would I add dynamic handler (Proxy address changes per request) to HttpClient? I don't see a way to add different handler values unless I initialize HttpClient in the method, because you can't add handlers after HttpClient is initialized

Pobiega•15mo ago

Use IHttpClientFactory, imho

Florian Voß•15mo ago

Great article, thx

Pobiega•15mo ago

well, this works fine. granted its only three urls and all three of those are usually quick to respond, but I'm 100% sure it started all three before the first one replied.

chef drone builder•15mo ago

what makes you think this?

Pobiega•15mo ago

yeah the whole "proxy addr changes per request" thing has me thrown off I've never ever had to do anything like that before.

chef drone builder•15mo ago

what repo?

Pobiega•15mo ago

its @Knuceles thread

Unknown User•15mo ago

Message Not Public

Pobiega•15mo ago

yeah actually, in all the samples they shown so far the URL seemed to be the same only the proxy changed.. so its hittiing the same url with multiple proxies

Unknown User•15mo ago

Message Not Public

Pobiega•15mo ago

Agreed

Unknown User•15mo ago

Message Not Public

Knuceles•15mo ago

What’s this about?

Unknown User•15mo ago

Message Not Public

Knuceles•15mo ago

What repository? The code related to proxies?

Unknown User•15mo ago

Message Not Public

Knuceles•15mo ago

Which thread

Unknown User•15mo ago

Message Not Public

Knuceles•15mo ago

Knuceles•15mo ago

Unknown User•15mo ago

Message Not Public

Knuceles•15mo ago

This is the thread related code

Unknown User•15mo ago

Message Not Public

Knuceles•15mo ago

I'm just making a proxy checker no i dont know what that means Regarding the google analytics cookies, i know that those cookies are generated from client-side javascript file, so I wanted to see how I can retrieve them with C# the intent is to see how I can get client-side js generated cookies

Unknown User•15mo ago

Message Not Public

Knuceles•15mo ago

Wdym?

Unknown User•15mo ago

Message Not Public

Knuceles•15mo ago

Yes, but there must be some way for me to mimic the browser behavior in C#, right?

Unknown User•15mo ago

Message Not Public

Knuceles•15mo ago

Obtaining the cookie somehow

Unknown User•15mo ago

Message Not Public

cap5lut•15mo ago

seems more like they want to bypass the openai rate limits, im out here

Unknown User•15mo ago

Message Not Public

Knuceles•15mo ago

Too meticulous

Knuceles•15mo ago

I'm just playing around with the OpenAI API if it's not too obvious, im clearly new to C# I created bunch of C# projects, and it's becoming too hard to manage, so I'm just putting everything in this one project

Unknown User•15mo ago

Message Not Public

Knuceles•15mo ago

which includes this proxy project

Accord•15mo ago

Was this issue resolved? If so, run /close - otherwise I will mark this as stale and this post will be archived until there is new activity.

Gaming

Programming

❔ Multithreading an async function