When piping into stdin of a non-C# program, need to detect when the program has ended
So I'm trying to make a small C# console application that creates an infinite loop to endlessly output raw binary data to stdout, in order for the data to be read by the stdin of a non-C# program, in this case, RNG_test of PractRand, a C++ program. I was able to make the program enter an infinite loop and send the data, by calling
Console.OpenStandardOutput() to get a Stream to write to. But the problem I'm having is that when I pipe the C# program's output to RNG_test, when RNG_test's process ends, the C# program, being in an infinite loop, continues to run until I kill it off with a Ctrl+C. I've been unable to find out if there is a way to determine if the C# program's stdout is still valid to write to or not. Is this possible?
As an aside, while searching for this I had come across the CliWrap project, which I thought could let me get around this by feeding a Stream into the wrapped process call and outputting the stdout/stderr of the wrapped process call as if I had called it directly, but I have not found a way to successfully do that. I've attempted to make an "echo" stream, which just stores a MemoryStream that fills when Write is called and send out the data when Read is called, but all this does is cause RNG_test to start and then apparently immediately stop, so I suspect I might not be sending the right data, but I'm not sure why. I don't really know of a good way to just make something like a pass-through Stream.11 Replies
Console.ReadLine will return null when it reaches the end of the stream.
I actually wasn't aware of the stream based Console methods. I would assume though that if you used the Console.In you would be able to detect the end of stream as you normally would.It appears you're implementing microservices. You mention difficulty knowing if one application is active. You also mention problems with infinite loops. I'd propose an event-driven system. This would mean your application runs once invoked with a request. Event driven architectures generaly have middleware in place such as kafka which help facilitate data exchange between apps.
So, firstly, I'm not using
Console.ReadLine, I'm using Stream's Write with a Span<byte> that is either 4 bytes or 8 bytes long (depending on if I am writing a 32-bit integer or a 64 bit-integer). Besides which, I'm writing to stdout, not reading from stdin. Also, no, this isn't a microservice. I only have control over the C# program, unless I actually modify the code of the accepting program (which I don't want to do, since I could use this piping C# program in more than just the one program). As such, I don't have event requests coming in. The accepting program is just reading from its stdin, which is coming from my C# program's stdout. So while I get what those suggestions are suggesting, neither one is valid for my purposes.
What I'm doing is basically trying to test C# versions of PRNGs, to do so, I am wanting to pipe the output of the PRNG directly into testing utilities, in this case, I'm piping into the stdin of PractRand's RNG_test, dieharder, and TestU01 via a program that reads from stdin. I could, in theory, modify the code of RNG_test but I would much prefer not to since it is not my code to begin with. I can't modify the code of dieharder. The TestU01 wrapper I could modify, but it is in C++, not C#, and as mentioned before, its only purpose is to read from stdin and run TestU01 on the incoming data.okay, so let me make sure I understand this correctly. You are doing something like:
And the issue is, you don't know when csharpProg should terminate?
I don't know if there is going to be a great way to do this, if you are doing the data transfer externally like this. Let me ponder.
Yes, that is correct on what I am doing.
I think the easiest way to handle this instead is have C# start up the cppProgram via um...
System.Diagnostic.Process that way you will have a handle to the external program which you can use to determine when the other program has finished.I tried to do that with CliWrap, but in order to write to the C++ program's stdin, I need to use a Stream and I don't think C# has any built-in Stream-derived types that can just take data being written to it and spit it back out when read from.
You could also maybe try and acquire the handle via other methods in the same namespace, but, I'm not 100% on how the invocation timing of all that works. That is, I don't know that if you launch the program via the pipe operator, when and if the pipe program will get executed so you can grab a handle to it.
so System.Diagnostic.ProcessInfo has like 16 billion options. But one of them is the StandardInput for the program you are launching, which you can feed a stream.
I could try just using a basic Process class, but in the morning, I gotta get some sleep. It's a bit of a hard thing to search Google for, either I get answers related to PipeStream (which is for named pipes and also not valid for my situation) or talking about the opposite situation (data being piped into the C# program instead of out) or talking about looking at Process's StandardOutput instead.
explore the namespace here: https://learn.microsoft.com/en-us/dotnet/api/system.diagnostics.processstartinfo.redirectstandardinput?view=net-8.0
you should find what you need.
I do think one of the only problems with that, at least with the fact that StandardInput is a StreamWriter instead of a Stream, is it makes it harder to do what I'm currently doing, since it treats the input stream as a character stream instead of a binary stream.
Although maybe the BaseStream of the StreamWriter would let me do it.
I'll mess with it in the morning.
So, what I've tried:
1. Making a
Process and redirecting its StandardInput. This technically works, but the downside is I get no output and if I try to redirect its StardardOutput, I end up in a state where it won't proceed because, as far as I can tell, because of how Process handles reading and writing from both input and output.
2. Going back to CliWrap, I found a set of stream from the Nerdbank.Streams package, so I used their SimplexStream. Results vary... The Task that is part of CliWrap's Command seems to be constantly in a state of WaitingForActivation even it somehow the process ends. And I'm not sure what combination of things I'm doing results in the C++ program even reacting to stuff on its stdin.
3. Also using CliWrap, I had to make a version of that SimplexStream that auto-flushes on each write, I got that idea from a GitHub issue on CliWrap. I also had to write my code so it would create the Task of the CliWrap command, created a Thread that runs the infinite loop, start the thread and immediately afterwards await the command. This works. If I use that as the pipe to stdin for the C++ program and then pipe its stdout to calling Console.Write, it works, but... output doesn't seem buffered very well. I was able to actually make PractRand's RNG_test work properly by adding a call of setvbuf(stdout, nullptr, _IONBF, 0); at the start of the program. I had to do the same for a TestU01 program. So maybe there is nothing that can be done on the C# side to get around the C++ program not buffering its output. But as far as outputting the C++ program's stdout to the C# program's stdout, what doesn't work is trying to copy to the stream I get from Console.OpenStandardOutput(). I instead have to pipe things to Console.WriteLine (or Console.Error.WriteLine for stderr).