C#•2y ago

❔ Stack Overflow Exception due to excessive cancellation token related method invocations?

I'm getting a stack overflow exception at unpredicable times and upon investigation in Rider's parallel stacks functionality I can see that pretty early on in my application I'm getting a rather verbose stack with a few calls related to cancellation token registration repeated over and over. What are some things I can begin investigating that might cause this behavior? For context, I have async/await processing in place, and several areas where I have tasks spawning tasks for example:

List<Task> processingTasks = new();
foreach (var sample in samples) {
    var task = _processor.Process(sample, cancellationToken);
    processingTasks.Add(task);
}

await Task.WhenAll(processingTasks);

// in processor.process
List<Task> generationTasks = new();
foreach (var foo in sample.Foos) {
    var task = _generator.Generate(foo, cancellationToken);
    generationTasks.Add(task);
}

await Task.WhenAll(generationTasks);

List<Task> processingTasks = new();
foreach (var sample in samples) {
    var task = _processor.Process(sample, cancellationToken);
    processingTasks.Add(task);
}

await Task.WhenAll(processingTasks);

// in processor.process
List<Task> generationTasks = new();
foreach (var foo in sample.Foos) {
    var task = _generator.Generate(foo, cancellationToken);
    generationTasks.Add(task);
}

await Task.WhenAll(generationTasks);

19 Replies

Hazel 🌊💃OP•2y ago

I'm having a difficult time adequately reproducing the exception, but I can clearly see there's bloat going on. The problem I'm having is determining what to look into based off of this limited information.

Hazel 🌊💃OP•2y ago

Here's the top of the large stack that will eventually overflow. Same repeating down the line. I think this is a path to investigate right now: https://source.dot.net/#Microsoft.Extensions.Configuration.FileExtensions/FileConfigurationProvider.cs,26 We have a hybrid atrocity for managing configurations and it has a problem where it's constantly updating configuration files. I've confirmed, it is the file configuration provider's on change functionality creating a stack overflow Even though the source is a bad design decision to support interim options I still think there needs to be visibility around the fact that it can lead to a stack overflow so that decisions can be made if it should be handled or not

JansthcirlU•2y ago

what happens when you return the Task.WhenAll(generationTasks) directly in Process(...) and let the outer await Task.WhenAll(processingTasks) handle everything?

Hazel 🌊💃OP•2y ago

That was a red herring I just supplied that because I had absolutely no idea where to get started When I slowed down and looked at the parallel stacks I noticed the common factor was file config provider

JansthcirlU•2y ago

oooh ok

Hazel 🌊💃OP•2y ago

Then started investigating further to find that our atrocity caused it to bloat the stack 🙂

this_is_pain•2y ago

wouldn't you be better of using dataflow? at least it would manage a bunch of things for you

Hazel 🌊💃OP•2y ago

With regards to?

this_is_pain•2y ago

in place of creating tasks and then calling Task.WhenAll (and eventually managing the cancellationToken)

Hazel 🌊💃OP•2y ago

I mean probably, but this issue had nothing to do with that in the end, I just gave some context because I didn't know where to start.

this_is_pain•2y ago

well it depends on the number of tasks you are creating, because dataflow would try to only create the "right" amount

Hazel 🌊💃OP•2y ago

Do you have a link for dataflow? Again, the task issue was a red herring The real issue here is that we have an atrocious options setup Where we are writing back to the observed files And FileConfigurationProvider has a recursive system in place to handle observations

this_is_pain•2y ago

it's this thing https://learn.microsoft.com/en-us/dotnet/standard/parallel-programming/dataflow-task-parallel-library why are configuration files being updated?

Dataflow (Task Parallel Library) - .NET

Learn how to use dataflow components in the Task Parallel Library (TPL) to improve the robustness of concurrency-enabled applications.

Hazel 🌊💃OP•2y ago

It's a bad interim solution to a longer-term problem. Thanks for the resource though 🙂

this_is_pain•2y ago

so you have an idea of how to approach this?

Hazel 🌊💃OP•2y ago

Yeah; I've got the interim fix in place and plan to report the stack overflow on dotnet runtime

Accord•2y ago

Was this issue resolved? If so, run /close - otherwise I will mark this as stale and this post will be archived until there is new activity.

Gaming

Programming

❔ Stack Overflow Exception due to excessive cancellation token related method invocations?

Did you find this page helpful?