WarChortle Posts - Answer Overflow

WarChortle

•Created by WarChortle on 9/14/2023 in #help

❔ Trying to maximize the amount of messages my worker can process while using Channels.

I am working with AWS SQS queue and using a channel to write messages to and read from. This is how I am writing them

while (!cancellationToken.IsCancellationRequested)
{
    if (await _channelWriter.WaitToWriteAsync(cancellationToken))
    {
        var messages = await _sqsClient.ReceiveMessageAsync(new ReceiveMessageRequest()
        {
            MaxNumberOfMessages = 10,
            QueueUrl = "https://sqs.us-east-1.amazonaws.com/563122478357/TestQueue"
        }, cancellationToken);

        foreach (var message in messages.Messages)
        {
            await _channelWriter.WriteAsync(message.Body, cancellationToken);
        }
    }
}

while (!cancellationToken.IsCancellationRequested)
{
    if (await _channelWriter.WaitToWriteAsync(cancellationToken))
    {
        var messages = await _sqsClient.ReceiveMessageAsync(new ReceiveMessageRequest()
        {
            MaxNumberOfMessages = 10,
            QueueUrl = "https://sqs.us-east-1.amazonaws.com/563122478357/TestQueue"
        }, cancellationToken);

        foreach (var message in messages.Messages)
        {
            await _channelWriter.WriteAsync(message.Body, cancellationToken);
        }
    }
}

and read them

while (!stoppingToken.IsCancellationRequested)
{
    //returns when a message exists to read from
    await _channelReader.WaitToReadAsync(stoppingToken);

    var message = await _channelReader.ReadAsync(stoppingToken);
    Console.WriteLine(message);
}

while (!stoppingToken.IsCancellationRequested)
{
    //returns when a message exists to read from
    await _channelReader.WaitToReadAsync(stoppingToken);

    var message = await _channelReader.ReadAsync(stoppingToken);
    Console.WriteLine(message);
}

My question is how will async work here. I am trying to maximize the concurrent execution of messages. Will C# just handle it for me, or do I have to do something to prevent a bunch of threads reading and 1 thread writing(causing starvation). Or vice versa?

2 replies

CC#

•Created by WarChortle on 4/10/2023 in #help

❔ IAsyncEnumerable vs List<T> as parameter in method

I have a method that is overloaded and will take in a List<T> or IAsyncEnumerable. The method will batch requests up to 10 then send them. I am confident the List<T> version is doing what I expect. But I am unsure exactly what to expect with the IAsyncEnumerable

await foreach (var message in messages)
        {
            var sqsMessage = JObject.Parse(message).ToObject<Message>();

            sendMessageBatchRequest.Entries.Add(new SendMessageBatchRequestEntry()
            {
                Id = sqsMessage?.MessageId,
                MessageBody = sqsMessage?.Body,
                MessageAttributes = sqsMessage?.MessageAttributes,
            });

            if (sendMessageBatchRequest.Entries.Count == batchSize)
            {
                failedMessages.AddRange(await SendMessagesAsync(sendMessageBatchRequest));
                sendMessageBatchRequest.Entries.Clear();

                if (_request.PublishingDelayInMs != null)
                {
                    await Task.Delay(_request.PublishingDelayInMs.Value);
                }
            }
        }

await foreach (var message in messages)
        {
            var sqsMessage = JObject.Parse(message).ToObject<Message>();

            sendMessageBatchRequest.Entries.Add(new SendMessageBatchRequestEntry()
            {
                Id = sqsMessage?.MessageId,
                MessageBody = sqsMessage?.Body,
                MessageAttributes = sqsMessage?.MessageAttributes,
            });

            if (sendMessageBatchRequest.Entries.Count == batchSize)
            {
                failedMessages.AddRange(await SendMessagesAsync(sendMessageBatchRequest));
                sendMessageBatchRequest.Entries.Clear();

                if (_request.PublishingDelayInMs != null)
                {
                    await Task.Delay(_request.PublishingDelayInMs.Value);
                }
            }
        }

The list T method is the same code but with foreach instead of await foreach.

5 replies

CC#

•Created by WarChortle on 2/8/2023 in #help

Looking for feedback on a plugin system, Specifically how each Plugin can define things for the app

I have an app that I am working on to make doing things at work quicker. I work with various AWS services and internal services and I wanted a simple front end to sped up minor things. Think Postman but not just http requests. I have a page you can add that lets you send messages to an SQS queue, I have a page that monitors GitHub Actions for a branch. Right now they are baked into the app, but I want to migrate to a plugin system eventually, so I am trying to figure out a way to make each view/viewmodel self sufficient. I am trying to define a common interface(s) for all these unrelated viewmodels for a few things things. 1. What to display in the Treeview. 2. What to display in a CardView. 3. How to define any inputs needed For the Treeview all views inherit from IDisplayableView which defines one property called Name, That is used by the treeview as what to click on An Example would be "MyQueue". For the CardView I have a custom class attribute called DisplayableAttribute it takes in 2 strings a Name (example SQS) so a generic name for the view and a string that is used to lookup images in a large nuget package. This creates a card. Lastly I am not sure on the best method to define what input a class needs. Each Class Needs at least a User Defined Name but might need more. SQS needs a URL, Github needs a branch name(soon to need repo name as well). The idea being that the Core app would load a plugin and when a user wanted to add that to the treeview it would generate a prompt asking for the input before passing it to the viewmodel. I was thinking tagging each property that the user needs to define with an InputAttribute or just having each view handle its own input in whatever way it deems, but I would like more consistency. I wanted to see what other ideas people had or thoughts on what I have so far.

2 replies

CC#

•Created by WarChortle on 9/28/2022 in #help

EFCore Updating an item after mapping with Automapper

Hey I am using Ef core to store some data in sqlite. I need to translate it to another object to show it in my UI. When I go to update it, it is adding another copy of the item to the DB. How can I transform between the two items and update or add as needed. How I am doing the update

public void UpsertAsync(TranslatedItem message)
{
    var result = _mapper.Map<EfItem>(message);
    if (_appDataContext.EnvItems.Any(x => x.Name == result.Name))
    {
        _appDataContext.EfItems.Update(result);
    }
    else
    {
        _appDataContext.EfItems.Add(result);
    }
    _appDataContext.SaveChanges();
}

public void UpsertAsync(TranslatedItem message)
{
    var result = _mapper.Map<EfItem>(message);
    if (_appDataContext.EnvItems.Any(x => x.Name == result.Name))
    {
        _appDataContext.EfItems.Update(result);
    }
    else
    {
        _appDataContext.EfItems.Add(result);
    }
    _appDataContext.SaveChanges();
}

public EfItem
{
  public int Id {get; set;}
  public string Name {get; set;}
}

public EfItem
{
  public int Id {get; set;}
  public string Name {get; set;}
}

public TranslatedItem : ReactiveObject
{
    private string _name;
    public string Name
    {
        get => _name;
            set => this.RaiseAndSetIfChanged(ref _name, value);
    }
}

public TranslatedItem : ReactiveObject
{
    private string _name;
    public string Name
    {
        get => _name;
            set => this.RaiseAndSetIfChanged(ref _name, value);
    }
}

2 replies

CC#

•Created by WarChortle on 9/26/2022 in #help

Creating a interface for a template engine for multiple different types

I created a very basic template engine to scan a string for a token and replace it. I am struggling to come up with an interface that can be used by various types to define what properties in it need to be passed through the template engine. The idea was to create a base class that any object could inherit from and by calling a method on that class all of the string properties(or all the defined string properties) could be run through the template engine. But I cant find a good way to do that. I looked into using reflection but wasn't able to get the value of the property. Any ideas on how to implement this, or on another approach.

4 replies

Gaming

Programming