C#•15mo ago

❔ Stack Overflow Exception due to excessive cancellation token related method invocations?

I'm getting a stack overflow exception at unpredicable times and upon investigation in Rider's parallel stacks functionality I can see that pretty early on in my application I'm getting a rather verbose stack with a few calls related to cancellation token registration repeated over and over. What are some things I can begin investigating that might cause this behavior? For context, I have async/await processing in place, and several areas where I have tasks spawning tasks for example:

List<Task> processingTasks = new();
foreach (var sample in samples) {
    var task = _processor.Process(sample, cancellationToken);
    processingTasks.Add(task);
}

await Task.WhenAll(processingTasks);

// in processor.process
List<Task> generationTasks = new();
foreach (var foo in sample.Foos) {
    var task = _generator.Generate(foo, cancellationToken);
    generationTasks.Add(task);
}

await Task.WhenAll(generationTasks);

List<Task> processingTasks = new();
foreach (var sample in samples) {
    var task = _processor.Process(sample, cancellationToken);
    processingTasks.Add(task);
}

await Task.WhenAll(processingTasks);

// in processor.process
List<Task> generationTasks = new();
foreach (var foo in sample.Foos) {
    var task = _generator.Generate(foo, cancellationToken);
    generationTasks.Add(task);
}

await Task.WhenAll(generationTasks);

19 Replies

Hazel 🌊💃OP•15mo ago

I'm having a difficult time adequately reproducing the exception, but I can clearly see there's bloat going on. The problem I'm having is determining what to look into based off of this limited information.

Hazel 🌊💃OP•15mo ago

Here's the top of the large stack that will eventually overflow. Same repeating down the line. I think this is a path to investigate right now: https://source.dot.net/#Microsoft.Extensions.Configuration.FileExtensions/FileConfigurationProvider.cs,26 We have a hybrid atrocity for managing configurations and it has a problem where it's constantly updating configuration files. I've confirmed, it is the file configuration provider's on change functionality creating a stack overflow Even though the source is a bad design decision to support interim options I still think there needs to be visibility around the fact that it can lead to a stack overflow so that decisions can be made if it should be handled or not

JansthcirlU•15mo ago

what happens when you return the Task.WhenAll(generationTasks) directly in Process(...) and let the outer await Task.WhenAll(processingTasks) handle everything?

Hazel 🌊💃OP•15mo ago

That was a red herring I just supplied that because I had absolutely no idea where to get started When I slowed down and looked at the parallel stacks I noticed the common factor was file config provider

JansthcirlU•15mo ago

oooh ok

Hazel 🌊💃OP•15mo ago

Then started investigating further to find that our atrocity caused it to bloat the stack 🙂

Omnissiah•15mo ago

wouldn't you be better of using dataflow? at least it would manage a bunch of things for you

Hazel 🌊💃OP•15mo ago

With regards to?

Omnissiah•15mo ago

in place of creating tasks and then calling Task.WhenAll (and eventually managing the cancellationToken)

Hazel 🌊💃OP•15mo ago

I mean probably, but this issue had nothing to do with that in the end, I just gave some context because I didn't know where to start.

Omnissiah•15mo ago

well it depends on the number of tasks you are creating, because dataflow would try to only create the "right" amount

Hazel 🌊💃OP•15mo ago

Do you have a link for dataflow? Again, the task issue was a red herring The real issue here is that we have an atrocious options setup Where we are writing back to the observed files And FileConfigurationProvider has a recursive system in place to handle observations

Omnissiah•15mo ago

it's this thing https://learn.microsoft.com/en-us/dotnet/standard/parallel-programming/dataflow-task-parallel-library why are configuration files being updated?

Dataflow (Task Parallel Library) - .NET

Learn how to use dataflow components in the Task Parallel Library (TPL) to improve the robustness of concurrency-enabled applications.

Hazel 🌊💃OP•15mo ago

It's a bad interim solution to a longer-term problem. Thanks for the resource though 🙂

Omnissiah•15mo ago

so you have an idea of how to approach this?

Hazel 🌊💃OP•15mo ago

Yeah; I've got the interim fix in place and plan to report the stack overflow on dotnet runtime

Accord•15mo ago

Was this issue resolved? If so, run /close - otherwise I will mark this as stale and this post will be archived until there is new activity.

Gaming

Programming

❔ Stack Overflow Exception due to excessive cancellation token related method invocations?