C#•7d ago

Race condition with Interlocked.Exchange

Hi, I found a weird bug which I can't quite understand... This is my class: https://github.com/AngryCarrot789/FramePFX/blob/master/FramePFX/Utils/RDA/RateLimitedDispatchAction.cs, it lets me add forced minimum interval between sequential executions (e.g. say you're dragging around a UI control but you only want to update the position text every say 0.1 seconds) InvokeAsync is called by the user and it choses whether to create a new task or set the current task to a "continue" state. ExecuteCore actually executes the action. At the very bottom is a generic parameter version, and at one point I hit a break point on the Debug.Assert(false) statement, which to me should be impossible, because ExecuteCore can only be executed if InvokeAsync has been invoked. And even if InvokeAsync gets invoked 10000 times before ExecuteCore, the Interlocked.Exchange should ensure that currentValue is swapped atomically... right? I rarely do multi threaded code so please tell me if i'm being turbo dumb and misusing Interlocked 😄

15 Replies

333fred•7d ago

And even if InvokeAsync gets invoked 10000 times before ExecuteCore, the Interlocked.Exchange should ensure that currentValue is swapped atomically... right?

Would it be possible to see the following order of operations? InvokeAsync InvokeAsync ExecuteCore ExecuteCore

bighugemassive3OP•7d ago

Not at all

333fred•7d ago

Fundamentally, in order to be observing null here, there must be a case where either you're observing the above ordering, or where ExecuteCore is being invoked before InvokeAsync Because you're correct, looking at the generic type at the bottom of the file, there is no way that you could observe a null in currentValue if it was always InvokeAsync then ExecuteAsync with no opportunity for overlap

bighugemassive3OP•7d ago

I just can't see how that could happen, unless i've completely missed a possible state somewhere ExecuteCore is invoked without the lock being taken, but the lock is taken before and after to check what's going on

333fred•7d ago

Given what you've said so far, I can safely say you missed a possible state 😄 I haven't looked through the rest of the file But I can see one of a few possibilities: 1. InvokeAsync starts, and then ExecuteCore starts before InvokeAsync is complete, and you observe the null that InvokeAsync hasn't yet set 2. Concurrent ExecuteAsyncs are running 3. ExecuteAsync starts before InvokeAsync

bighugemassive3OP•7d ago

Hm interesting

bighugemassive3OP•7d ago

The breakpoint didn't hit but that that one case did happen

333fred•7d ago

You're at the mercy of thread suspension there Both do the exchange as the first operation, and the second invoke and first execute happened at basically the same point If the breakpoint didn't hit, then probably the execute won that race Seems like that's possibility 2 though

bighugemassive3OP•7d ago

So there could be 2 tasks somehow? I don't see how it's possible though, a new task is only started if S_RUNNING is not present, and S_RUNNING is removed from the state when the current task is just about to exit All done with the lock taken

333fred•7d ago

I mean, assuming that those logs were all the same RateLimitedDispatchAction instance, then yes, you have proof there are I haven't scrutinized the rest of your code though, so I can't comment on the rest of it

bighugemassive3OP•7d ago

I don't blame you it's horrid Oh wait they are different instances lol

bighugemassive3OP•7d ago

Based on GetHashCode

333fred•7d ago

Are you certain that ExecuteCore is complete before InvokeAsync is called? Those Invoke/Execute pairs are extremely close together

bighugemassive3OP•7d ago

InvokeAsync could get called at any time so not really It's why I added the lock so that I could at least get a slice of time to sync everything InvokeAsync could get called just before or just after ExecuteCore is invoked, and even just before and just after it completes Which is why I introduced the critical continue state, which tells the task to keep running since InvokeAsync was invoked very closely around ExecuteCore Annoyingly I can't hit the break point with some Debug.WriteLine code added Nevermind, just hit it and i actually did get that case you spoke about

8730397  | 19867.92867 ExecuteCore with FramePFX.Utils.RDA.RateLimitedDispatchAction`1+ObjectWrapper[FramePFX.Editing.UI.ITimelineElement]
8730397  | 19868.05789 InvokeAsync
8730397  | 19868.05834 ExecuteCore with FramePFX.Utils.RDA.RateLimitedDispatchAction`1+ObjectWrapper[FramePFX.Editing.UI.ITimelineElement]
8730397  | 19868.05949 InvokeAsync
8730397  | 19868.19200 InvokeAsync
8730397  | 19868.19279 InvokeAsync
8730397  | 19868.19279 ExecuteCore with FramePFX.Utils.RDA.RateLimitedDispatchAction`1+ObjectWrapper[FramePFX.Editing.UI.ITimelineElement]
8730397  | 19868.31126 ExecuteCore with

8730397  | 19867.92867 ExecuteCore with FramePFX.Utils.RDA.RateLimitedDispatchAction`1+ObjectWrapper[FramePFX.Editing.UI.ITimelineElement]
8730397  | 19868.05789 InvokeAsync
8730397  | 19868.05834 ExecuteCore with FramePFX.Utils.RDA.RateLimitedDispatchAction`1+ObjectWrapper[FramePFX.Editing.UI.ITimelineElement]
8730397  | 19868.05949 InvokeAsync
8730397  | 19868.19200 InvokeAsync
8730397  | 19868.19279 InvokeAsync
8730397  | 19868.19279 ExecuteCore with FramePFX.Utils.RDA.RateLimitedDispatchAction`1+ObjectWrapper[FramePFX.Editing.UI.ITimelineElement]
8730397  | 19868.31126 ExecuteCore with

InvokeAsync called 3 times sequentially, ExecuteCore called twice sequentially Looks like InvokeAsync was called in the same 10~ microsecond window as ExecuteCore I guess InvokeAsync updated the value after the first ExecuteCore, and then that cleared the value. But because InvokeAsync was called during the first ExecuteCore, the critical state got set so ExecuteCore was called again I'm thinking this might do the trick

ObjectWrapper? value;
lock (this.stateLock)
{
    value = Interlocked.Exchange(ref this.currentValue, null);
    this.ClearCriticalState();
}

ObjectWrapper? value;
lock (this.stateLock)
{
    value = Interlocked.Exchange(ref this.currentValue, null);
    this.ClearCriticalState();
}

Clear the critical state in case InvokeAsync is called within microseconds Appreciate the help fred 👍

Gaming

Programming

Race condition with Interlocked.Exchange