C#•3y ago

How to locally loadbalance threadsafe [Answered]

Hola, I got an instance of a class that holds a list of end-points. The class performs and awaits an http request to one of the end-points in the list. The same instance of the class will be accessed async from multiple threads (about 300 times a second) I need to round-robin load-balance the endpoints in the list. Initially I thought of using a concurrentQueue and to dequeue/enqueue before and after every request. but the locking will hurt my performance as I have no need to wait for each request to finish before using the same end-point again.. I want that eventually within the 1 second, the 300 requests will be balanced between all the endpoints without delaying them. I thought of using counter field as an index accessor, that I keep adjusting its value in a circle according to the end-points length. what do you think? is there a better approach? Simplified example -

public class MyClass
{
  List<string> _endpoints;
  int _indexCounter = 0;
  public MyClass(List<string> endpoints)
  {
    _endpoints = endpoints;    
  }
  
  public Task SendRequest()
  {
    var endpoint = _endpoints[_indexCounter];
    indexCounter = _indexCounter < _endpoints.Length-1 ? _indexCounter+1 : 0;
    
    await client.Request(endpoint);
  }
}

public class MyClass
{
  List<string> _endpoints;
  int _indexCounter = 0;
  public MyClass(List<string> endpoints)
  {
    _endpoints = endpoints;    
  }
  
  public Task SendRequest()
  {
    var endpoint = _endpoints[_indexCounter];
    indexCounter = _indexCounter < _endpoints.Length-1 ? _indexCounter+1 : 0;
    
    await client.Request(endpoint);
  }
}

9 Replies

Xymanek•3y ago

I would take a similar approach to what you suggested with index incrementing, but use Interlocked.Increment and then modulo to get the index of endpoint to use so something like

private readonly IReadOnlyList<string> _endpoints;
private ulong _indexCounter = 0;

public Task SendRequest()
{
  ulong newCounter = Interlocked.Increment(ref _indexCounter);
  int index = (int) (newCounter % (ulong) _endpoints.Count);

  await client.Request(_endpoints[index]);
}

private readonly IReadOnlyList<string> _endpoints;
private ulong _indexCounter = 0;

public Task SendRequest()
{
  ulong newCounter = Interlocked.Increment(ref _indexCounter);
  int index = (int) (newCounter % (ulong) _endpoints.Count);

  await client.Request(_endpoints[index]);
}

(I switched the counter from int to ulong to avoid overflow issues)

BenOP•3y ago

Ok so instead of reseting the indexCounter you modulo it with the length of the array. makes sense, not really worried about overflow each instance won't be surpassing the 1-2 million. Using Interlocked.Increment means that a lock is in place on every update so it makes the indexCounter incremental thread-safe, correct? It shouldn't affect performance as such a basic operation won't hold the lock? That being said, I can abandon the index counter and use concurrent queue to manage that for me if I dequeue and enqueue before awating the request? Something like -

  ConcurrentQueue<string> _endpoints;
  public MyClass(List<string> endpoints)
  {
    _endpoints = new ConcurrentQueue<string>(endpoints);
  }
  
  public Task SendRequest()
  {
    var endpoint = _endpoints.Dequeue();
    _endpoints.Enqueue(endpoint);
    
    await client.Request(endpoint);
  }

  ConcurrentQueue<string> _endpoints;
  public MyClass(List<string> endpoints)
  {
    _endpoints = new ConcurrentQueue<string>(endpoints);
  }
  
  public Task SendRequest()
  {
    var endpoint = _endpoints.Dequeue();
    _endpoints.Enqueue(endpoint);
    
    await client.Request(endpoint);
  }

What do you think? @Xymanek

Xymanek•3y ago

Using Interlocked.Increment means that a lock is in place on every update

no Interlocked.xyz methods are atomic I suggested it precisely since there is no locking

MODiX•3y ago

❯ : System.Threading.Interlocked.Increment(Int32) Increments a specified variable and stores the result, as an atomic operation. ❯ : System.Threading.Interlocked.Increment(Int64) Increments a specified variable and stores the result, as an atomic operation. ❯ : System.Threading.Interlocked.Increment(UInt32) Increments a specified variable and stores the result, as an atomic operation. 3/4 results shown ~ Click Here for more results

React with ❌ to remove this embed.

Xymanek•3y ago

as an atomic operation

any queue approach will definitely be more expensive than increment with modulo

BenOP•3y ago

Ok so queue is out. I'm not sure what atomic operation actually is, not sure how it achieves that without a lock. I'll be googling for a few minutes and come back afterwards 😅 Saying that an atomic operation is having a lock on the hardware level is a correct statement?

Xymanek•3y ago

ehhhh in terms of the effect - yes, your statement is correct in terms of how it works - not correct

BenOP•3y ago

Got it. ok makes sense now. Thanks mate! I'll close this topic

Accord•3y ago

✅ This post has been marked as answered!

Gaming

Programming

How to locally loadbalance threadsafe [Answered]