C#•2y ago

✅ Task not returning and stopping program

Hey, I am currently trying to start one task for every string in the list, but this isn't working and the program closes without actually calling the requested method nor displaying an error message. Any help appreciated

var tasks = links.Select(x => Task.Run(() => Scraper.ScrapFromLink(x)));
            await Task.WhenAll(tasks);

var tasks = links.Select(x => Task.Run(() => Scraper.ScrapFromLink(x)));
            await Task.WhenAll(tasks);

38 Replies

JimmahDean•2y ago

how is the method being called?

Jona4PlayOP•2y ago

This is the definition

private static async Task<List<Document>> Scraping(IEnumerable<string> links)

private static async Task<List<Document>> Scraping(IEnumerable<string> links)

And it's being called as part of this foreach

foreach (var list in partionedLists)
                    {
                        var scrapingTask = await Task.Run(() => Scraping(list));

foreach (var list in partionedLists)
                    {
                        var scrapingTask = await Task.Run(() => Scraping(list));

JimmahDean•2y ago

i need more context 99% of the time this problem is because main isn't awaiting the function call and just returns

Jona4PlayOP•2y ago

Control Point B is never printed

private static async Task<List<Document>> Scraping(IEnumerable<string> links)
        {
            Console.WriteLine("Control Point A");
            var tasks = links.Select(x => Task.Run(() => Scraper.ScrapFromLink(x)));
            await Task.WhenAll(tasks);
            Console.WriteLine("Control Point B");
            
            var documents = new List<Document>();
            
            foreach (var task in tasks)
            {
                Task.Run(() => PostgreSQL.PushDocument(task.Result));
                documents.Add(task.Result);
            }
            
            Console.WriteLine("Leaving Scraping Method");
            return documents;
        }

private static async Task<List<Document>> Scraping(IEnumerable<string> links)
        {
            Console.WriteLine("Control Point A");
            var tasks = links.Select(x => Task.Run(() => Scraper.ScrapFromLink(x)));
            await Task.WhenAll(tasks);
            Console.WriteLine("Control Point B");
            
            var documents = new List<Document>();
            
            foreach (var task in tasks)
            {
                Task.Run(() => PostgreSQL.PushDocument(task.Result));
                documents.Add(task.Result);
            }
            
            Console.WriteLine("Leaving Scraping Method");
            return documents;
        }

JimmahDean•2y ago

right i need to see how this method is called. the problem is not in that method

sibber•2y ago

didnt you ask the same question yesterday?

JimmahDean•2y ago

yes

sibber•2y ago

also you want to ToArray() this without that it will run the tasks each time you enumerate it

Jona4PlayOP•2y ago

This is the part where the method is called in Main()

if (outstanding != null)
                {
                    var partionedLists = outstanding.Partition(SimultaneousPool);
                    Console.WriteLine("Partitioned Lists: " + partionedLists.Count());
                    foreach (var list in partionedLists)
                    {
                        var scrapingTask = await Task.Run(() => Scraping(list));

                        foreach (var document in scrapingTask)
                        {
                            if (document.Links != null)
                            {
                                outstandingLinks.AddRange(document.Links);
                            }
                            if (document.PrioritisedLinks != null)
                            {
                                prioritisedLinks.AddRange(document.PrioritisedLinks);
                            }
                        }
                    }

if (outstanding != null)
                {
                    var partionedLists = outstanding.Partition(SimultaneousPool);
                    Console.WriteLine("Partitioned Lists: " + partionedLists.Count());
                    foreach (var list in partionedLists)
                    {
                        var scrapingTask = await Task.Run(() => Scraping(list));

                        foreach (var document in scrapingTask)
                        {
                            if (document.Links != null)
                            {
                                outstandingLinks.AddRange(document.Links);
                            }
                            if (document.PrioritisedLinks != null)
                            {
                                prioritisedLinks.AddRange(document.PrioritisedLinks);
                            }
                        }
                    }

Yes 😅 but I haven't made much progress hence

JimmahDean•2y ago

Control Point A is printing?

Jona4PlayOP•2y ago

Yes I have tried splitting the LINQ statement into a foreach and the whole program exits after trying to start the task Which is weird as it runs perfectly fine synchronously

JimmahDean•2y ago

Might be a dumb question but are you running this with a debugger

Jona4PlayOP•2y ago

I mean I'm using Visual Studio So yeah Should I switch to release?

JimmahDean•2y ago

are you running it with the VS debugger though? it won't necessarily show you errors if you're not usually it does, but i've had occasions where things just die for no reason oh wait var scrapingTask = await Task.Run(() => Scraping(list)); this should be var scrapingTask = await Scraping(list); the way you have it awaits the start of Scraping which happens instantly and then it continues on

Jona4PlayOP•2y ago

nope. That doesn't seem to be the problem

sibber•2y ago

again your enumerating twice here runnning the scraping twice use ToArray()

Jona4PlayOP•2y ago

I have added it This is now of type Task<Document>[]

JimmahDean•2y ago

what does this mean

Jona4PlayOP•2y ago

Tested it and it still only reaches Point A

sibber•2y ago

why arent your db calls async youre not awaiting them either

Jona4PlayOP•2y ago

The error seems to be

await Task.WhenAll(tasks);

await Task.WhenAll(tasks);

sibber•2y ago

all your IO stuff should be async youre wasting threads by running them sync then using Task.Run and cluttering your code and making it more complex for no reason

JimmahDean•2y ago

something's missing here can i get the entirety of Main()

sibber•2y ago

i really suggest you refactor all that then come back and implement this properly

Jona4PlayOP•2y ago

Ok so I'd make them async and await them? I don't see why to await them. I just want to fire the push and then move on Might be a good idea

JimmahDean•2y ago

fire and forget async code is a very bad idea, especially for io

sibber•2y ago

you dont care if the push finishes? or if it failed?

Jona4PlayOP•2y ago

It has honestly become quite cluttered when implementing async as it was an afterthought

JimmahDean•2y ago

that will happen

sibber•2y ago

it shouldnt have been an afterthought, youre doing a lot of IO

Jona4PlayOP•2y ago

Yeah. This started out as a small project. I will need to take some time of weekend and not only rewrite the DBManager but also Scraper itself

sibber•2y ago

yeah you should do that first

Jona4PlayOP•2y ago

well. Thank you for your guidance. Do you know any useful guides to implementing it besides the MSDN article.

JimmahDean•2y ago

there's a stackoverflow regular that has a lot of really good blog posts about async stuff let me see if i can find his name

JimmahDean•2y ago

https://blog.stephencleary.com/ you'll have to search around for async specific stuff but what i've found of his has been very useful for me

Stephen Cleary (the blog)

Stephen Cleary's blog: async/await, programming, language design, and other sundry computer science topics.

JimmahDean•2y ago

https://blog.stephencleary.com/2013/10/taskrun-etiquette-and-proper-usage.html etc

Task.Run Etiquette and Proper Usage

I have a confession to make: I enjoy etiquette. My wife and I own a good dozen or two etiquette books, ranging from the classic Post to a more playful Austen-themed guide as well as several historical books (including an uncut copy of Social Life of Virginia in the Seventeenth Century). There’s a certain nerdy appreciation of knowing how to act ...

Jona4PlayOP•2y ago

Thanks. I will look into this! I'm going to close this thread then. Thanks for your help

Accord•2y ago

Closed!

Gaming

Programming

✅ Task not returning and stopping program