C#•15mo ago

❔ Processes and Async

How can I recreate this with Async? (BeginOutputReads and such)

// If an error happens, prints it and returns 1.
            if (proc.StandardError.Peek() != -1)
            {
                Console.Clear(); Console.ForeColor = Color.Firebrick;

                while (!proc.StandardError.EndOfStream)
                    Console.AppendText(Logging.LOGGER.Error(await proc.StandardError.ReadLineAsync()) + Environment.NewLine);

                return 1;
            }
            
            while (!proc.StandardOutput.EndOfStream)
            {
                // Read the output line by line, and wait for the "Done" line to be printed.
                await Task.Delay(100);
                
                string line = await proc.StandardOutput.ReadLineAsync();
                Logging.LOGGER.Info(line);
                if (line != null && !line.Contains("Done")) continue;
                
                Console.AppendText(Logging.LOGGER.Info("Server is DONE, killing process") + Environment.NewLine);
                break;
            }

// If an error happens, prints it and returns 1.
            if (proc.StandardError.Peek() != -1)
            {
                Console.Clear(); Console.ForeColor = Color.Firebrick;

                while (!proc.StandardError.EndOfStream)
                    Console.AppendText(Logging.LOGGER.Error(await proc.StandardError.ReadLineAsync()) + Environment.NewLine);

                return 1;
            }
            
            while (!proc.StandardOutput.EndOfStream)
            {
                // Read the output line by line, and wait for the "Done" line to be printed.
                await Task.Delay(100);
                
                string line = await proc.StandardOutput.ReadLineAsync();
                Logging.LOGGER.Info(line);
                if (line != null && !line.Contains("Done")) continue;
                
                Console.AppendText(Logging.LOGGER.Info("Server is DONE, killing process") + Environment.NewLine);
                break;
            }

52 Replies

Alex•15mo ago

https://paste.md-5.net/ipewadetam.cs for clarity

JakenVeina•15mo ago

uhh you mean you want to move AWAY from async/await? any particular reason?

Alex•15mo ago

I want to move into async await Since this apparently isn't it from what I've seen

JakenVeina•15mo ago

uhh no

Alex•15mo ago

Because this deadlocks

JakenVeina•15mo ago

you are using async/await now THAT is a different story let's see, what's the intention here?

Alex•15mo ago

For it not to deadlock; Online I've seen it be said that I need to use BeginOutputReads and BeginErrorReads, which fire events every time that there's a new stdout/stderr to be read. However, I have no idea if I could have the same logic as I have up there with events.

JakenVeina•15mo ago

I mean

Alex•15mo ago

Anddd everyone was calling THAT async, so i got confused lol

JakenVeina•15mo ago

"async" and async/await are not the same thing async/await is a type of "async" programming in any case what I mean is what is the intention of your code what are you trying to do

Alex•15mo ago

When reading stderr, I'm trying to check if anything at all is written to it, and if so, log it and return so the method doesn't continue. Then it moves onto reading STDOUT, which just logs everything until it finds the word "Done" in there, and then it kills the process because it's not needed anymore

JakenVeina•15mo ago

so, I'll assume proc is a Process?

Alex•15mo ago

The issues that can arise from STDERR happen at the start so yeah there's no need to keep checking Yep

JakenVeina•15mo ago

you've started up an external process, and told it run and you want to read from it

Alex•15mo ago

yeah

JakenVeina•15mo ago

where does the "deadlock" occur?

Alex•15mo ago

By redirecting both STDOUT and STDERR, the internal buffer of the process gets filled, and then when we read anything it tries to write into it, and causes a deadlock because it'll wait for it to be free before anything else It's documented on MSDN as a known issue, and recommends using BeginOutputReads + ErrorReads

JakenVeina•15mo ago

ah, okay well, there's a decent chance that's what the async methods are using under the hood so, it's a deadlock within the external process?

Alex•15mo ago

Yep. It'll lock until the buffer is somehow unfilled (Which it never won't because the program is frozen) and by consequence... We won't be able to read from it

JakenVeina•15mo ago

the buffer that the process is writing STDOUT and STDERR to

Alex•15mo ago

So we won't go through with the program And ours will hang too

JakenVeina•15mo ago

and reading from STDOUT and STDERR by the parent process requires calling into the child process I feel like you might be misunderstanding the issue er the MSDN recommendation it's not the fault of the parent process to change how it calls into the child process to read those buffers it's the fault of the child process for deadlocking I.E. it's the child process that should be implementing a way to read the buffers that doesn't deadlock by, say, having a separate thread for console I/O the main thread that's trying to write to those buffers can block until there's some space, but it should be a separate thread that is responsible for reading from them got a link?

Alex•15mo ago

Yeah that would be neat in a perfect world but the stuff that I'm trying to run does it in the same thread, so it's up to me to go around it To?

JakenVeina•15mo ago

the MSDN documentation like, if the child process is trying to use the same thread for both reading and writing those buffers, I don't think there's a damn thing the parent process can do except "try your best to read from the child process fast enough that the buffers don't fill up"

Alex•15mo ago

Ahh okay A sec

Alex•15mo ago

https://learn.microsoft.com/en-us/dotnet/api/system.diagnostics.processstartinfo.redirectstandardoutput?redirectedfrom=MSDN&view=net-7.0#System_Diagnostics_ProcessStartInfo_RedirectStandardOutput

ProcessStartInfo.RedirectStandardOutput Property (System.Diagnostics)

Gets or sets a value that indicates whether the textual output of an application is written to the StandardOutput stream.

Alex•15mo ago

That's probably why they tell you to use BeginOutputReads then, it just reads it instantly But I can't return my method from an event can I?

JakenVeina•15mo ago

alrighty, so this is something that you're doing within the parent? you're redirecting the child's output into a stream you can read from?

Alex•15mo ago

Can i change variables that my method has access to from the events? Nope, I'm currently doing as described above

JakenVeina•15mo ago

but, like so, you're currently creating the child process with RedirectStandardOutput = false? surely not, right? that's required for the parent to be able to read from the child at all, right?

Alex•15mo ago

Nope, it's set to true The issue isn't there, it's just on the deadlock

JakenVeina•15mo ago

okay, cool

Alex•15mo ago

This would probably solve my entire thing to be fair But its weird

JakenVeina•15mo ago

I really have no idea what that means what events?

Alex•15mo ago

BeginOutputReads and BeginErrorReads assign methods to be executed with every new line that appears Oj either stderr or stdout

JakenVeina•15mo ago

okay alright, so you're launching this child process and you want to read from both STDOUT and STDERR you want to terminate the process early if it has any errors, yes? after logging those you also want to terminate the process early as soon as you see "Done" in the output

Alex•15mo ago

Correct

JakenVeina•15mo ago

so, this isn't about deadlocking or asynchronism it's about parallelism you need to be doing two things at once: listening for data on two streams, and be able to respond accordingly, when some input is received

Alex•15mo ago

I was reading it synchronously and not in parallel, so it was deadlocking If the issue can be solved with doing things in parallel, yep. That's probably what BeginOutputReads does

JakenVeina•15mo ago

yeah, sort of if you were going to set both BeginOutputReads and BeginErrorReads at the same time, then yeah there really isn't any deadlock that I see in your little snippet unless there's a .WaitForExit() call that's not shown

process.Start();

async Task ListenForErrorsAsync()
{
    await process.StandardError.ReadLineAsync();
    process.Terminate();
}

async Task ListenForDoneAsync()
{
    await foreach(var line in process.StandardOutput.ReadAllLinesAsync())
    {
        if (line.Contains("Done"))
            break;
    }
    process.Terminate();
}

var whenFailed = ListenForErrorsAsync();
var whenDone   = ListenForDoneAsync();

await Task.WhenAny(whenFailed, whenDone);

if (whenDone.IsComplete)
    ...
else if (whenFailed.IsCompleted)
    ...

process.Start();

async Task ListenForErrorsAsync()
{
    await process.StandardError.ReadLineAsync();
    process.Terminate();
}

async Task ListenForDoneAsync()
{
    await foreach(var line in process.StandardOutput.ReadAllLinesAsync())
    {
        if (line.Contains("Done"))
            break;
    }
    process.Terminate();
}

var whenFailed = ListenForErrorsAsync();
var whenDone   = ListenForDoneAsync();

await Task.WhenAny(whenFailed, whenDone);

if (whenDone.IsComplete)
    ...
else if (whenFailed.IsCompleted)
    ...

I feel like this could be a bit simpler, but I might just be too tired to think it through functionally, this is about what you want although, I am making a couple assumptions about how the STD I/O streams work mainly, that they should close when you manually terminate a process if not, you might need to toss in a CancellationTokenSource that you can use to manually cancel whichever read doesn't complete first also, like, you probably want some null-checks and stuff in there like, on the ReadLine() calls the non-async/await version of this would look something like

process.ErrorDataReceived => (sender, e) => process.Terminate();
process.OutputDataReceived => (sender, e) => 
{
    if (e.Data.Contains("Done"))
        process.Terminate();
    else
        process.BeginOutputReadLine();
};

process.Start();

process.BeginErrorReadLine()
process.BeginOutputReadLine();
process.WaitForExit();

process.ErrorDataReceived => (sender, e) => process.Terminate();
process.OutputDataReceived => (sender, e) => 
{
    if (e.Data.Contains("Done"))
        process.Terminate();
    else
        process.BeginOutputReadLine();
};

process.Start();

process.BeginErrorReadLine()
process.BeginOutputReadLine();
process.WaitForExit();

or, actually this ought to work too

process.ErrorDataReceived => (sender, e) => process.Terminate();

process.Start();

process.BeginErrorReadLine()
string? line;
while((line = process.StandardOutput.ReadLine()) is not null)
{
    if (line.Contains("Done"))
        process.Terminate();
}

process.ErrorDataReceived => (sender, e) => process.Terminate();

process.Start();

process.BeginErrorReadLine()
string? line;
while((line = process.StandardOutput.ReadLine()) is not null)
{
    if (line.Contains("Done"))
        process.Terminate();
}

although, I would guess that this might throw an exception out of .ReadLine() if the error handler calls process.Terminate()

Alex•15mo ago

Whilst search of solutions I actually stumbled upon something similar, and it still hangs on process.StandardOutput.ReadAllLinesAsync() and the other one. I am not really able to explain very well why but it does. To be fair I probably didn't explain how the process hangs properly but oh well

JakenVeina•15mo ago

if it does hang, it's because process.StandardOutput doesn't close itself when you manually terminate the process

Alex•15mo ago

How would I differentiate between termination from ErrorDataReceived and OutputDataReceived?

JakenVeina•15mo ago

which doesn't really make any sense to me

Alex•15mo ago

¯\_(ツ)_/¯

JakenVeina•15mo ago

however you like they're 2 different events and you have complete control over them

Alex•15mo ago

int terminationCode;

process.ErrorDataReceived => (sender, e) =>
{ 
  process.Terminate();
  terminationCode = 1;
}

process.OutputDataReceived => (sender, e) => 
{
    if (e.Data.Contains("Done")) {
        process.Terminate();
        terminationCode = 0;
    }
    else
        process.BeginOutputReadLine();
};

process.Start();

process.BeginErrorReadLine()
process.BeginOutputReadLine();
process.WaitForExit();

int terminationCode;

process.ErrorDataReceived => (sender, e) =>
{ 
  process.Terminate();
  terminationCode = 1;
}

process.OutputDataReceived => (sender, e) => 
{
    if (e.Data.Contains("Done")) {
        process.Terminate();
        terminationCode = 0;
    }
    else
        process.BeginOutputReadLine();
};

process.Start();

process.BeginErrorReadLine()
process.BeginOutputReadLine();
process.WaitForExit();

Would this work? I'm not sure if you can just do it like that

JakenVeina•15mo ago

other than the fact that 1 and 0 are not string, sure

Alex•15mo ago

yeah, I haven't had my coffee yet Oh, so events declared in that way run within the scope? nice Well then, thank you!

JakenVeina•15mo ago

anonymous delegates can capture items from the scope, yes it's called a Closure the compiler takes care of it all for you it creates an anonymous class to contain both the method and whatever values are captured, and rewrites your code to create an instance of that class instead of the relevant variables

Alex•15mo ago

ahh i see thank you!

Accord•15mo ago

Was this issue resolved? If so, run /close - otherwise I will mark this as stale and this post will be archived until there is new activity.

Gaming

Programming

❔ Processes and Async