C#•4w ago

Named Pipe IPC: Protocols & Status Codes

I'm basically starting from scratch with my implementation, since I've gone way too deep with my initial approach and need something that I can actually finally work with. I'm writing an IPC implementation via named pipes. I'm open to other ideas (I don't know if anonymous pipes are appropriate here), but named pipes are definitely what I've settled on. I cannot use additional NuGet packages and I'm on .NET Standard 2.0 (C# 13, PolySharp). My requirements are the following: I will have multiple server implementations. Each server fulfills a different purpose and has multiple methods (I will call them endpoints) pertaining to that purpose. A server endpoint can be the equivalent to one of these: Action, Action<T>, Func<TResult>, Func<T, TResult>. To clarify; an endpoint can optionally receive some request data, and can optionally return some response data. Each server will be paired up with a client implementation that should simplify calling server endpoints for the user. These clients should return some Result type to signal success or failure. If the server responds with a failure, the kind should be communicated to the client. This includes unhandled server exceptions. My idea was to send data via JSON. The client serializes the request; sends it to the server; the server deserializes the data and calls the corresponding endpoint implementation. I'm not sure how to handle figuring out this correspondence. I'm looking for some ideas on how to implement this in a manner that is reasonably robust and can be extended to more server-client-pairs without many problems.

160 Replies

eroOP•4w ago

cc @Tanner Gooding @cap5lut if you'd like to take a peek i'm gonna provide some more implementation ideas in a bit and you can tell me what i should do instead :p To be a bit less vague, this is about game interop. Unity, Unreal, GameMaker, etc. I need a different server-client-pair for each of those. To determine the correspondence between the request data the server receives and the actual handler for that endpoint, I was thinking of simply using an enum or multipe enums. The thing that's in the way here is that all servers share some base requests, like Close. This makes using multiple enums a bit cumbersome. Should I use strings instead? In terms of responses; I was initially using a different enum type for every endpoint in the server. This makes the entire implementation a big mess and was my main source of issues before. I could possibly use integer status codes, but I'm afraid that will fall flat when it comes to unhandled server exceptions. Should I just send plain strings containing the reason for failure?

canton7•4w ago

My idea was to send data via JSON

If you can't use nuget packages, and you don't have S.T.J, then the only json serializer you have is pretty rubbish 😛

eroOP•4w ago

oh, i can use that one actually. i use source generation with it. some others that i can use (provided by the app that i'm writing this plugin for) are System.Memory and System.Buffers, for example

canton7•4w ago

Is there a fully SG'd json library which doesn't need any runtime libraries at all? Which one?

eroOP•4w ago

no no i just use stj

canton7•4w ago

Ah no, I see what you mean

eroOP•4w ago

i can technically use any package i il weave everything but i really want to keep the file size low

canton7•4w ago

It looks like you've got a few different problems here: 1. Defining the different messages types 2. Mapping messages types into RPC methods to call 3. Serializing exceptions Correct?

eroOP•4w ago

i think that's a good summary

canton7•4w ago

How are you defining your schema/interface? I.e. what messages can be sent between a given server and client? (also note that json-rpc is a thing. Might be worth looking at what they do) The normal way to approach something like this is that you define your schema centrally (i.e. what the message types are, what the parameters are, etc), then code-generate the server and client from that schema The code generation handles converting method call -> message on the sender, and message -> method call on the receiver

eroOP•4w ago

i'm not sure how i should define them. simply the question of whether to use interfaces or not, whether to use classes or structs... some of the data i would send would be like

record GetMonoImageRequest(
  string Name);

record GetMonoImageResponse(
  ulong Address,
  string Name,
  string ModuleName,
  string FilePath);

record GetMonoImageRequest(
  string Name);

record GetMonoImageResponse(
  ulong Address,
  string Name,
  string ModuleName,
  string FilePath);

i really don't wanna write a source generator for this, and rpc libraries tend to be pretty large

canton7•4w ago

Yeah, they're large for a reason! Chucking messages back and forth is a lot easier

eroOP•4w ago

it's unfortunately not an option for me. i really want the download of the plugin to be quick i mean it's an option, but not the first i'd like to choose

canton7•4w ago

Just to play devil's advocate -- can't you include any large libraries in your host application?

eroOP•4w ago

the host application isn't mine

canton7•4w ago

Ah, your IPC isn't with the host application?

eroOP•4w ago

the ipc is between my library (loaded by the host app) and some game i'm essentially writing both sides, though

canton7•4w ago

Ah gotcha Can you just pass messages around, rather than going full-on RPC?

eroOP•4w ago

that's exactly what i want to do the server side of course needs handlers

canton7•4w ago

I thought you said you were turning method calls -> messages on the sender, and turning a message -> a method call on the receiver, with automatically serializing exceptions from that method call, etc?

eroOP•4w ago

well, i thought i'd have some abstract base client, which contains the raw logic for sending over the request. there would then be several concrete clients which contain methods that hide this raw logic:

class MonoClient : IpcClient
{
  public Result<GetMonoImageResponse> GetMonoImage(string name)
  {
    GetMonoImageRequest request = new(name);
    return base.CallServerEndpoint(request); // or some such
  }
}

class MonoClient : IpcClient
{
  public Result<GetMonoImageResponse> GetMonoImage(string name)
  {
    GetMonoImageRequest request = new(name);
    return base.CallServerEndpoint(request); // or some such
  }
}

where IpcClient.CallServerEndpoint would serialize the request and just send it over the pipe

canton7•4w ago

Yeah, it's the other side which gets a bit painful -- working out what the message type is, mapping that to a method to call, extracting the parameters, marshalling exceptions back, making sure that the response ends up in the same place that the request came from. Do-able, but boilerplatey if you're writing it by hand. At that point, you might as well just deal with the messages directly Anyway, none of this is answering your original questions, I think Ah no, it does cover:

I'm not sure how to handle figuring out this correspondence.

The short answer is: 1. Lots of hand-written boilerplate 2. Some method to register a method to a message type, but which reduces some of the boilerplate but probably introduces some runtime reflection / codegen 3. Compile-time code generation from a schema 4. Don't, and deal with the messages directly (4 is what I do, FWIW)

cap5lut•4w ago

u basically have this: 1) transport (fixed via named pipes i guess) 2) serialization (a bit of a struggle but u do it somehow) 3) application layer protocol 4) business logic 3 seems to be what u are stuck at, right?

eroOP•4w ago

i've basically nuked the entire project :p so i'm at 0

cap5lut•4w ago

well, then u have these 4 layers to work on 😂

eroOP•4w ago

which is what this thread is about :)

cap5lut•4w ago

well, with 1 and 2 there arent problems are there?

eroOP•4w ago

just in the sense that i was looking for possible alternatives if they make the process easier, more performant, more consistent, more in line with modern technologies

cap5lut•4w ago

im not sure what ya mean by that. 1) would mean that u transfer a bunch of bytes, 2) would mean that u interpret that bunch of bytes. "consistency and modern technologies" would be mostly in the application and business layer the (de)serialization of 2) as well tho, but thats usually not the problematic thing

this_is_pain•4w ago

i still don't understand what your fear is about implementing this, or what is it that you don't know that require so much to think about

eroOP•4w ago

You don't know how deep in the mud I was with my attempted implementation It wasbad bad. I'll share some of it tomorrow if you care Well things like, do I use messagepack instead of json, do I use some custom format I implement myself, do I use anonymous pipes instead of named ones, do I use tcp, mem mapped files, mutexes, etc

cap5lut•4w ago

i would abstract the transport layer and data (de)serialization away, so that u can first use simple stuff, like tcp and json and then make it configurable for faster alternatives like messagepack and named pipes/shared memory and the actual application layer protocol for rpc u can build on top of that

this_is_pain•4w ago

i understand but for example how can you still be undecided about ipc vs tcp? given that as long your wrappers implements Stream they should be interchangeable, they are for two different use cases anyway, local vs remote communcation also the approach to serialization and designing (for the higher level protocol) would change if we are talking about modeling 2 structures or 10000; maybe it could change even between 1000 and 10000; it's kind of an important information to have

eroOP•4w ago

well "ipc vs tpc" doesn't even make sense in the first place, no? you can do ipc via tcp. or via named pipes. or via websockets. or via memory mapped files. or... and i mean, that's what this thread is for, to have people ask clarifying questions like this :p there won't be too many structures to model. i figure 5-20 per server implementation (of which there will be ~3-5)

cap5lut•4w ago

the number of different high level data types doesnt matter, their serialization boils down to the primitives you want to support. at the transport layer you are just sending bytes over, maybe with some sequence and fragment number to support something like UDP where packets could arrive out of order, or simply dont arrive. the "hard" part here is only to discern which bytes belong to which message once thats done and dusted it goes to how to do the serialization, no matter if thats reflection based or source generated or manually writing it, which itself isnt that problematic either, and can also be abstracted away. ones all that is fixed u just need the application layer protocol, where the hard part is designing it. basically here the different types come into play, for requests, for responses. u want some generic status codes + optional data to process messages, pretty much like HTTP status codes.

this_is_pain•4w ago

the number of different high level data types doesnt matter

i think it could matter, depending on the amount of effort required and tooling available i think it makes sense as a question because if you write a non generic ipc in your service then to transform it to work between different machines instead of locally you have to refactor it or reengineer it, so it would be better if you knew that beforehand didn't you say some time ago that you had a lot of these messages to send/receive? or was it another issue

eroOP•2w ago

i'm not sure i remember that i still don't have this figured out i keep trying to rewrite it, but i just end up in the same situations i don't know how to do this

canton7•2w ago

What are you having trouble with?

eroOP•2w ago

genuinely i don't even know how to start anymore at this point. i've been on and off trying to find a good implementation for this for months months! my brain is completely fried when it comes to this project i do not know what to do i don't even know what i'm envisioning anymore i don't even know where to begin explaining it like i want something like this or something

interface IMonoServer
{
  IpcResponse<MonoImageInfo> GetMonoImage(string name);
}

abstract class IpcServer<TServer>
{
  // some way to receive requests, send responses
}

class MonoServer : IpcServer<IMonoServer>, IMonoServer
{
  public IpcResponse<MonoImageInfo> GetMonoImage(string name)
  {
    // call mono api
    // return error status code or actual response object
  }
}

abstract class IpcClient<TServer>
{
  // some way to send requests, receive responses
}

class MonoClient : IpcClient<IMonoServer>
{
  public IpcResponse<MonoImageInfo> GetMonoImage(string name)
  {
    return base.Server.GetMonoImage(string name); // or idek
  }
}

interface IMonoServer
{
  IpcResponse<MonoImageInfo> GetMonoImage(string name);
}

abstract class IpcServer<TServer>
{
  // some way to receive requests, send responses
}

class MonoServer : IpcServer<IMonoServer>, IMonoServer
{
  public IpcResponse<MonoImageInfo> GetMonoImage(string name)
  {
    // call mono api
    // return error status code or actual response object
  }
}

abstract class IpcClient<TServer>
{
  // some way to send requests, receive responses
}

class MonoClient : IpcClient<IMonoServer>
{
  public IpcResponse<MonoImageInfo> GetMonoImage(string name)
  {
    return base.Server.GetMonoImage(string name); // or idek
  }
}

i don't even know anymore i don't know what to do even if i were to drop this coupling, i have no idea how to send the data i need naot json serialization so what do i just send

struct Request
{
  int Code;
  object? Data;
}

struct Request
{
  int Code;
  object? Data;
}

? and then the other side has to read

struct Request
{
  int Code;
  JsonElement Data;
}

struct Request
{
  int Code;
  JsonElement Data;
}

? i can't use the latter on both sides, because jsonserializer would try to serialize the JsonElement struct and its internals i'm completely lost

this_is_pain•2w ago

i imagined this thread would wake up again why naot? but you don't need to use JsonElement on both side, you use the correct models/dto for the job JsonElement would be a step for the deserializer to get to the correct model

eroOP•2w ago

I need to pass information about what "endpoint" I'm calling (just a string, or an enum) as well as optionally the request model. That means I need a type that cannot be generic (what generic arg would I pass if I'm not passing a model?), but still contains both the endpoint identifier and some data property (null if not specified). The other side then needs to read that data property as a JsonElement and check whether it's null or something Because the server side is a library loaded into another process for communication. Of course the library needs to be native to be loaded

canton7•2w ago

Doesn't STJ have support for $type? Yes it does, with JsonDerivedType I assume that works with naot As I said before, you'll have a much easier time if you think in terms of sending messages around rather than method calls: doing RPC adds quite a lot of overhead

this_is_pain•2w ago

i don't know, it looks to me that you're worrying/focusing too much about the technicalities and the 'rules' (like json is text, you can send almost whatever you want) and not on what these actions/messages are used for; by that i mean structure follows function, if you need nested types you nest types, if you need 200 flat fields you make a flat class with 200 nullable primitive fields; if generics don't work, don't use it, or maybe you can have two discriminators instead of just one (or honestly i think there are better ways, usually); the more practical you can be the more we can help i guess

eroOP•2w ago

@this_is_pain @canton7 i don't think either of you understand what the problem is? i need to send request types like this:

struct Request
{
  string Action;
}

struct Request<T>
{
  string Action;
  T Data;
}

struct Request
{
  string Action;
}

struct Request<T>
{
  string Action;
  T Data;
}

the other side needs to read this structure, but it of course needs to do so in one go. reading

struct Request
{
  string Action;
}

struct Request
{
  string Action;
}

would drop the data that may or may not be there. reading

struct Request
{
  string Action;
  JsonElement Data;
}

struct Request
{
  string Action;
  JsonElement Data;
}

would over-read if the first struct above was sent, leading to an exception.

canton7•2w ago

I don't think you read my response...

eroOP•2w ago

please go ahead and explain how $type would help here

canton7•2w ago

That's literally the problem that $type is there to solve

eroOP•2w ago

please, have at it don't think i need to mention the obvious, but requests do not derive from one another there is absolutely no relation between

record GetFooRequest(
  string Bar,
  long Baz);

record GetFooRequest(
  string Bar,
  long Baz);

and

record GetQuxRequest(
  int[] Quo);

record GetQuxRequest(
  int[] Quo);

canton7•2w ago

Let's say you have two message types, SayHelloRequest and SayGoodbyeRequest. You could define your RequestMessage message as:

[JsonDerivedType(typeof(SayHelloRequest), "sayHello")]
[JsonDerivedType(typeof(SaGoodbyeRequest), "sayGoodbye")]
public class RequestPayloadBase { }

public class SayHelloRequestPayload : RequestPayloadBase  { ... }
public class SayGoodbyeRequestPayload : RequestPayloadBase  { ... }

public class RequestMessage
{
    // Any common fields here
    RequestPayloadBase Payload { get; set; }
}

[JsonDerivedType(typeof(SayHelloRequest), "sayHello")]
[JsonDerivedType(typeof(SaGoodbyeRequest), "sayGoodbye")]
public class RequestPayloadBase { }

public class SayHelloRequestPayload : RequestPayloadBase  { ... }
public class SayGoodbyeRequestPayload : RequestPayloadBase  { ... }

public class RequestMessage
{
    // Any common fields here
    RequestPayloadBase Payload { get; set; }
}

Then:

var requestMessage = new RequestMessage()
{
    Payload = new SayHelloRequestPayload(...),
}

var requestMessage = new RequestMessage()
{
    Payload = new SayHelloRequestPayload(...),
}

Gets serialized as:

{
    "payload": {
        "$type": "sayHello",
        // Other fields from SayHelloRequestPayload 
    }
}

{
    "payload": {
        "$type": "sayHello",
        // Other fields from SayHelloRequestPayload 
    }
}

don't think i need to mention the obvious, but requests do not derive from one another

Maybe that's why you're causing yourself so much trouble 🤷‍♂️ You're necessarily doing polymorphic messages. STJ requires that your different message types inherit from a common base. That's how it works. If you're insisting on doing something which STJ supports, but without following its requirements, yes you're going to have a hard time. I'm not sure what to say in response to that.

eroOP•2w ago

and explain how you would receive this on the other side

canton7•2w ago

Just deserialize as RequestMessage, then type switch over Payload

eroOP•2w ago

and explain how you would handle request messages that don't have a payload

canton7•2w ago

Then they have an empty payload. Every message needs a type.

eroOP•2w ago

wdym type switch? how do i access $type?

canton7•2w ago

STJ creates the correct type in the Payload member. If you serialized a SayHelloRequestPayload, that gets deserialized as a SayHelloRequestPayload So you can do switch (message.Payload) { case SayHelloRequestPayload sayHello: ... }, or use the visitor pattern, or a Dictionary<Type, ...>, or whatever

eroOP•2w ago

sorry if i'm a bit agitated, i've just been trying things for way too long and i'm mostly upset with myself for not trying this before so request messages without data still need some way to be distinguished think of it like Func<TRet> or Action no input, but (optionally) some output

canton7•2w ago

☝️

eroOP•2w ago

well yeah, but i still need some identifier

canton7•2w ago

The identifier is the type

eroOP•2w ago

if there's no payload, there's no type

canton7•2w ago

Which is why you need a payload, even an empty one I feel like we're going in circles here

eroOP•2w ago

an empty payload sounds like a dumb idea no offense

canton7•2w ago

As I said before, don't think in terms of calling methods or RPC or similar: think in terms of messages I mean, I'm literally telling you how this is done. If you want to think you know better and invent your own system, go ahead, but please don't ask us for help (and bearing in mind how much trouble you're having inventing your own system, taking a lead from how a lot of other people solve this might not be the worst idea) (You can use a singleton instance of the payload if you want, although STJ will still try to deserialize it as a new instance)

eroOP•2w ago

i guess... it just seems so different from what i would usually do alright well, i think i might be able to do something with that now for the handling and responses on the server side how would you "register" the handlers?

canton7•2w ago

The simplest is to literally have a big switch statement which switches over all possible request message types, and calls an appropriate function in each case. If you want to make it a bit more decoupled, you could have a Dictionary<Type, Action<RequestPayloadBase>> where you register delegates which handle each message type

eroOP•2w ago

i think i would like to have some base server (just handles receiving basic requests and sending basic responses), a... more concrete abstract class(?) that registers handlers and defines simpler abstract methods for each "endpoint", and then the actual server implementation:

class Server
{
  // send, receive
}

abstract class FooServer : Server
{
  // register handlers

  abstract Result<BarResponse> HandleBar(BarRequest request);
}

sealed class FooServerImpl : FooServer
{
  override Result<BarResponse> HandleBar(BarRequest request) => throw null;
}

class Server
{
  // send, receive
}

abstract class FooServer : Server
{
  // register handlers

  abstract Result<BarResponse> HandleBar(BarRequest request);
}

sealed class FooServerImpl : FooServer
{
  override Result<BarResponse> HandleBar(BarRequest request) => throw null;
}

god damn it yeah i've been doing that

canton7•2w ago

Or yeah, you can start using reflection to discover handler methods, etc, but that becomes a bit more magic. I like explicitness.

eroOP•2w ago

god damn it yes i've realized man i love the magic but i want tight coupling like a proxy interface would be kinda sick but like how do you even make that good

canton7•2w ago

Tbh, I'd probably use the visitor pattern. It's a bit of boilerplate, but a lot of stuff just falls out

eroOP•2w ago

do you have a link or an example i haven't heard of it

canton7•2w ago

public interface IRequestMessageVisitor
{
    void Accept(SayHelloRequestPayload payload);
    void Accept(SayGoodbyeRequestPayload payload);
}

public class RequestPayloadBase
{
    public abstract void Visit(IRequestMessageVisitor visitor);
}

public class SayHelloRequestPayload : RequestPayloadBase 
{
    public override void Visit(IRequestMessageVisitor visitor) => visitor.Accept(this);
}

public class SayGoodbyeRequestPayload : RequestPayloadBase
{
    public override void Visit(IRequestMessageVisitor visitor) => visitor.Accept(this);
}

public interface IRequestMessageVisitor
{
    void Accept(SayHelloRequestPayload payload);
    void Accept(SayGoodbyeRequestPayload payload);
}

public class RequestPayloadBase
{
    public abstract void Visit(IRequestMessageVisitor visitor);
}

public class SayHelloRequestPayload : RequestPayloadBase 
{
    public override void Visit(IRequestMessageVisitor visitor) => visitor.Accept(this);
}

public class SayGoodbyeRequestPayload : RequestPayloadBase
{
    public override void Visit(IRequestMessageVisitor visitor) => visitor.Accept(this);
}

Then in the receiver:

public class FooServer : IRequestMessageVisitor
{
    public void ReceiveMessage(RequestMessage message) => message.Payload.Visit(this);

    void Accept(SayHelloRequestPayload payload) { ... }
    void Accept(SayGoodbyeRequestPayload payload) { ... }
}

public class FooServer : IRequestMessageVisitor
{
    public void ReceiveMessage(RequestMessage message) => message.Payload.Visit(this);

    void Accept(SayHelloRequestPayload payload) { ... }
    void Accept(SayGoodbyeRequestPayload payload) { ... }
}

Implementing the Visit method on all of the payload types is a little painful, but tbh VS generates most of it for you, and you get compiler errors unless/until you define everything, which is really nice -- there's no way to accidentally forget to support a particular message type

eroOP•2w ago

i am truly shocked at this design and that i'm not smart enough to have come up with that tunnel vision i mean that's crazy

canton7•2w ago

Visitor's one of those patterns that you probably wouldn't come up with yourself, but it's really useful when you have a known set of types, and you want to find out what you've got, and process it, in a strongly-typed way It's less relevant now that C# has switch over types, but it's still useful because it guarantees that you've handled all cases (you can make a visitor base class with virtual methods if you want implementors to be able to pick and choose what they handle: ExpressionVisitor does that for instance)

eroOP•2w ago

yeah fuck that's crazy smart so simple very effective, thanks

canton7•2w ago

So, empty payload types do make sense 🙂

eroOP•2w ago

alright well, i need some response type next i want to communicate different types of failures like something is not found, something was null

canton7•2w ago

Yeah. Start with messages again, then see how you can work them into your visitor stuff Tbh I like just having a separate call to send a response message, rather than forcing it through a method's return type, but whatever, both work

eroOP•2w ago

i would need to see both i think

canton7•2w ago

public class FooServer : IRequestMessageVisitor
{
    public void ReceiveMessage(RequestMessage message)
    {
        ResponseMessage response = new(...);
        try
        {
            response.Payload = msesage.Payload.Visit(this);
        }
        catch (Exception ex)
        {
            response.Error = ...;
        }
        // Send / return response
    }

    SayHelloResponsePayload Accept(SayHelloRequestPayload payload) { ... }
    SayGoodbyeResponsePayload Accept(SayGoodbyeRequestPayload payload) { ... }
}

public class FooServer : IRequestMessageVisitor
{
    public void ReceiveMessage(RequestMessage message)
    {
        ResponseMessage response = new(...);
        try
        {
            response.Payload = msesage.Payload.Visit(this);
        }
        catch (Exception ex)
        {
            response.Error = ...;
        }
        // Send / return response
    }

    SayHelloResponsePayload Accept(SayHelloRequestPayload payload) { ... }
    SayGoodbyeResponsePayload Accept(SayGoodbyeRequestPayload payload) { ... }
}

Maybe? The rest is left as an exercise to the reader

eroOP•2w ago

no god mutable type

canton7•2w ago

The other option is:

public class FooServer : IRequestMessageVisitor
{
    public void ReceiveMessage(RequestMessage message) => message.Payload.Visit(this);

    void Accept(SayHelloRequestPayload payload)
    {
        // Do stuff
        client.SendResponse(new SayHelloResponsePayload(...));
    }
    void Accept(SayGoodbyeRequestPayload payload) { ... }
}

public class FooServer : IRequestMessageVisitor
{
    public void ReceiveMessage(RequestMessage message) => message.Payload.Visit(this);

    void Accept(SayHelloRequestPayload payload)
    {
        // Do stuff
        client.SendResponse(new SayHelloResponsePayload(...));
    }
    void Accept(SayGoodbyeRequestPayload payload) { ... }
}

Serialization models can be mutable: that's fine. But make it immutable if you want, there's no difference You could have the error be a separate field on ResponseMessage, or you could make an error response be one of the possible ResponseMessagePayload subclasses. Both work. You probably want to have some sort of correlation token to tie together a request and response. The sender puts some integer value as the correlation token in the request, and the receiver mirrors that same value back in the response. That lets the sender tie together the response with the request that they sent (Which is important with errors, as otherwise you don't necessarily know what request message an error is in response to) Alternatively you could put the error on ResponsePayloadBase No right or wrong answers there (Then on the sender side you have a Dictionary<int, TaskCompletionSource> which maps together a correlation token with a TCS. That TCS is what the sender is awaiting, and you complete it when you get a response back with the matching correlation token) Does that all make sense?

eroOP•2w ago

I'm taking a little break, I'll catch up on it later, thanks :) One thing though, I was looking to design it in such a way that the payload base, the serialization code and the pipe stream writing code is contained in one project, and the actual implementations (visitor, client, server, payloads) are in separate projects (one project per "kind of server"). How would you design this generically? I haven't yet figured out a good way

canton7•2w ago

That's fine? You can split this up into: * RequestMessage / ResponseMessage (these can be generic over the base payload type, or you can use inheritance) * The networking code to transmit RequestMessage / ResponseMessage, retries, etc * A particular payload and all of its subclasses * The types which implement IRequestMessageVisitor etc

eroOP•2w ago

i can't figure it out Proj1

public interface IRequest<in TVisitor>
{
    void Visit(TVisitor visitor);
}

public sealed class RequestMessage // generic how?
{
    // ?
}

// some base server idk
public class Server<TVisitor> // or no? or maybe we need TRequestVisitor and TResponseVisitor?
    // maybe constrain to some empty IRequestVisitor interface?
{
    // what methods? abstract? virtual?
}

public interface IRequest<in TVisitor>
{
    void Visit(TVisitor visitor);
}

public sealed class RequestMessage // generic how?
{
    // ?
}

// some base server idk
public class Server<TVisitor> // or no? or maybe we need TRequestVisitor and TResponseVisitor?
    // maybe constrain to some empty IRequestVisitor interface?
{
    // what methods? abstract? virtual?
}

Proj2

public interface IFooRequestVisitor
{
    void Accept(GetFooRequest request);
}

public sealed class GetFooRequest : IRequest<IFooRequestVisitor>
{
    public void Visit(IFooRequestVisitor visitor)
    {
        visitor.Accept(this);
    }
}

public sealed class FooServer : Server<IFooRequestVisitor>, IFooRequestVisitor // ???
{
    public void Accept(GetFooRequest request)
    {

    }

    // response? idk
}

public interface IFooRequestVisitor
{
    void Accept(GetFooRequest request);
}

public sealed class GetFooRequest : IRequest<IFooRequestVisitor>
{
    public void Visit(IFooRequestVisitor visitor)
    {
        visitor.Accept(this);
    }
}

public sealed class FooServer : Server<IFooRequestVisitor>, IFooRequestVisitor // ???
{
    public void Accept(GetFooRequest request)
    {

    }

    // response? idk
}

public sealed class RequestMessage<TRequest, TVisitor>(TRequest request)
    where TRequest : IRequest<TVisitor>
{
    public TRequest Payload { get; } = request;
}

public sealed class RequestMessage<TRequest, TVisitor>(TRequest request)
    where TRequest : IRequest<TVisitor>
{
    public TRequest Payload { get; } = request;
}

public class Server<TVisitor>(TVisitor visitor)
{
    public void HandleRequest<TRequest>(RequestMessage<TRequest, TVisitor> request)
        where TRequest : IRequest<TVisitor>
    {
        request.Payload.Visit(visitor);
    }
}

public class Server<TVisitor>(TVisitor visitor)
{
    public void HandleRequest<TRequest>(RequestMessage<TRequest, TVisitor> request)
        where TRequest : IRequest<TVisitor>
    {
        request.Payload.Visit(visitor);
    }
}

(mind the primary ctors) but passing TVisitor to the base ctor of Server<TVisitor> seems wrong when the implementing server is itself the visitor especially since

public FooServer()
    : base(this) { }

public FooServer()
    : base(this) { }

is not even valid... (for obvious reasons)

canton7•2w ago

One way:

public class RequestMessage<TRequestPayload>
{
    // ...
    public TRequestPayload Payload { get; set; }
}

public abstract class ServerBase<TRequestPayload, TResponsePayload>
{
    public abstract void HandleMessage(RequestMessage<TRequestPayload> message);
}

public class RequestMessage<TRequestPayload>
{
    // ...
    public TRequestPayload Payload { get; set; }
}

public abstract class ServerBase<TRequestPayload, TResponsePayload>
{
    public abstract void HandleMessage(RequestMessage<TRequestPayload> message);
}

Then in proj2:

public interface IRequestPayloadVisitor
{
    void Accept(SayHelloRequestPayload payload);
    void Accept(SayGoodbyeRequestPayload payload);
}

[JsonDerivedType(typeof(SayHelloRequestPayload), "sayHello")]
[JsonDerivedType(typeof(SayHelloRequestPayload), "sayGoodbye")]
public class RequestPayloadBase { }

public class SayHelloRequestPayload : RequestPayloadBase { ... }

public class FooServer : ServerBase<RequestPayloadBase, ...>, IRequestPayloadVisitor
{
    public override void HandleMessage(RequestMessage<RequestPayloadBase> message) => message.Payload.Visit(this);

   // IRequestPayloadVisitor impl
}

public interface IRequestPayloadVisitor
{
    void Accept(SayHelloRequestPayload payload);
    void Accept(SayGoodbyeRequestPayload payload);
}

[JsonDerivedType(typeof(SayHelloRequestPayload), "sayHello")]
[JsonDerivedType(typeof(SayHelloRequestPayload), "sayGoodbye")]
public class RequestPayloadBase { }

public class SayHelloRequestPayload : RequestPayloadBase { ... }

public class FooServer : ServerBase<RequestPayloadBase, ...>, IRequestPayloadVisitor
{
    public override void HandleMessage(RequestMessage<RequestPayloadBase> message) => message.Payload.Visit(this);

   // IRequestPayloadVisitor impl
}

Or you could do:

public abstract class RequestMessageBase
{
    // ...
}

public abstract class ServerBase<TRequestMessage, TResponseMessage> where TRequestMessage : RequestMessageBase
{
    public abstract void HandleMessage(TRequestMessage message);
}

public abstract class RequestMessageBase
{
    // ...
}

public abstract class ServerBase<TRequestMessage, TResponseMessage> where TRequestMessage : RequestMessageBase
{
    public abstract void HandleMessage(TRequestMessage message);
}

Then in proj2:

public interface IRequestPayloadVisitor
{
    void Accept(SayHelloRequestPayload payload);
    void Accept(SayGoodbyeRequestPayload payload);
}

[JsonDerivedType(typeof(SayHelloRequestPayload), "sayHello")]
[JsonDerivedType(typeof(SayHelloRequestPayload), "sayGoodbye")]
public class RequestPayloadBase { }

public class SayHelloRequestPayload : RequestPayloadBase { ... }

public class FooRequestMessage : RequestMessageBase
{
    public RequestPayloadBase Payload { get; set; }
}

public class FooServer : ServerBase<FooRequestMessage, ...>, IRequestPayloadVisitor
{
    public override void HandleMessage(FooRequestMessage message) => message.Payload.Visit(this);

   // IRequestPayloadVisitor impl
}

public interface IRequestPayloadVisitor
{
    void Accept(SayHelloRequestPayload payload);
    void Accept(SayGoodbyeRequestPayload payload);
}

[JsonDerivedType(typeof(SayHelloRequestPayload), "sayHello")]
[JsonDerivedType(typeof(SayHelloRequestPayload), "sayGoodbye")]
public class RequestPayloadBase { }

public class SayHelloRequestPayload : RequestPayloadBase { ... }

public class FooRequestMessage : RequestMessageBase
{
    public RequestPayloadBase Payload { get; set; }
}

public class FooServer : ServerBase<FooRequestMessage, ...>, IRequestPayloadVisitor
{
    public override void HandleMessage(FooRequestMessage message) => message.Payload.Visit(this);

   // IRequestPayloadVisitor impl
}

Or you could do something in the middle:

public abstract class RequestMessageBase<TRequestPayload>
{
    // ...
    public TRequestPayload Payload { get; set; }
}

public abstract class ServerBase<TRequestMessage, TResponseMessage> where TRequestMessage : RequestMessageBase
{
    public abstract void HandleMessage(TRequestMessage message);
}

public abstract class RequestMessageBase<TRequestPayload>
{
    // ...
    public TRequestPayload Payload { get; set; }
}

public abstract class ServerBase<TRequestMessage, TResponseMessage> where TRequestMessage : RequestMessageBase
{
    public abstract void HandleMessage(TRequestMessage message);
}

Then in proj2:

public interface IRequestPayloadVisitor
{
    void Accept(SayHelloRequestPayload payload);
    void Accept(SayGoodbyeRequestPayload payload);
}

[JsonDerivedType(typeof(SayHelloRequestPayload), "sayHello")]
[JsonDerivedType(typeof(SayHelloRequestPayload), "sayGoodbye")]
public class RequestPayloadBase { }

public class SayHelloRequestPayload : RequestPayloadBase { ... }

public class FooRequestMessage : RequestMessageBase<RequestPayloadBase> { }

public class FooServer : ServerBase<FooRequestMessage, ...>, IRequestPayloadVisitor
{
    public override void HandleMessage(FooRequestMessage message) => message.Payload.Visit(this);

   // IRequestPayloadVisitor impl
}

public interface IRequestPayloadVisitor
{
    void Accept(SayHelloRequestPayload payload);
    void Accept(SayGoodbyeRequestPayload payload);
}

[JsonDerivedType(typeof(SayHelloRequestPayload), "sayHello")]
[JsonDerivedType(typeof(SayHelloRequestPayload), "sayGoodbye")]
public class RequestPayloadBase { }

public class SayHelloRequestPayload : RequestPayloadBase { ... }

public class FooRequestMessage : RequestMessageBase<RequestPayloadBase> { }

public class FooServer : ServerBase<FooRequestMessage, ...>, IRequestPayloadVisitor
{
    public override void HandleMessage(FooRequestMessage message) => message.Payload.Visit(this);

   // IRequestPayloadVisitor impl
}

eroOP•2w ago

i'm not following these examples. they all have a server which only handles... one message? i need the server to handle an arbitrary amount of requests

canton7•2w ago

Yes, you're reading them wrong (or I'm not being clear enough)

eroOP•2w ago

i mean i'm probably reading them wrong

canton7•2w ago

Where do you think we're only handling one message type? In all cases, proj1 defines the RequestMessage but not the payload type, and has logic to handle the RequestMessage itself, but not the individual payload types. Then proj2 introduces the different payload types and how to handle those

eroOP•2w ago

oh yeah you define RequestPayloadBase in proj 2, that's exactly not what i want RequestPayloadBase should be in proj 1 it's the base after all

canton7•2w ago

Re-read it Wait Why do you want RequestPayloadBase in each project? I thought you wanted the individual payload types to be defined in proj2?

eroOP•2w ago

i don't, that's what you wrote >Then in proj2 >public class RequestPayloadBase

canton7•2w ago

and the actual implementations (visitor, client, server, payloads) are in separate projects (one project per "kind of server").

payloads

eroOP•2w ago

i mean just look at the code you wrote idk? what else should i say lol

canton7•2w ago

I'm saying that you wanted the payloads to be defined in proj2. You said so. If you don't want that, then I'm confused as to what you want

eroOP•2w ago

yes, but not the payload base it's the base, all other projects need it

canton7•2w ago

Ok, stop. You're blindly wanting the impossible again Remember, the payload base is something that STJ needs. It's part of the mechanism for doing polymorphism in STJ You need to add attributes to the payload base to tell STJ what all of the different payload types are If you put the payload base in proj1, then all of the different payload types would also, obviously, have to be in proj1. Because otherwise how could you name them in the attributes on the payload base So you can put the request message in proj1, but if the individual payloads need to be in proj2, then the payload base also has to be in proj2 (Note that the payload base is probably empty. It's literally just a marker class, and a place to stick attributes)

eroOP•2w ago

but then where is Payload.Visit coming from

canton7•2w ago

What do you mean? Ah yeah, it does implement the visitor pattern, that's right. I forgot that bit

eroOP•2w ago

alright

canton7•2w ago

But that needs to be in proj2 as well, because the visitor interface is specific to the different payload types, and they're in proj2 too You could put this in proj1:

public abstract class RequestMessagePayloadBaseBase<TVisitor>
{
    public void Visit(TVisitor visitor);
}

public abstract class RequestMessagePayloadBaseBase<TVisitor>
{
    public void Visit(TVisitor visitor);
}

Then in proj2:

public interface IRequestPayloadVisitor { ... }
public class RequestMessagePayloadBase : RequestMessagePayloadBaseBase<IRequestPayloadVisitor> { }

public interface IRequestPayloadVisitor { ... }
public class RequestMessagePayloadBase : RequestMessagePayloadBaseBase<IRequestPayloadVisitor> { }

But I really don't see the point

eroOP•2w ago

oh JsonDerivedTypeAttribute doesn't exist that's really bad

canton7•2w ago

.NET 7 apparently? Or NS2.0 with a nuget package

eroOP•2w ago

so i'm on uhh 2.0 :/

canton7•2w ago

NS2.0? Or .net core 2.0?

eroOP•2w ago

canton7•2w ago

eroOP•2w ago

i'm on a pretty old version of the package (don't remember why) probably compatibility with other packages (System.Memory, System.Buffers)

canton7•2w ago

You can probably write a custom serializer which does the same thing?

eroOP•2w ago

i'm not all that picky, but it seems a tall order to do what stj does

canton7•2w ago

I mean https://learn.microsoft.com/en-us/dotnet/standard/serialization/system-text-json/converters-how-to

eroOP•2w ago

that's for converters?

canton7•2w ago

That's what I meant -- used the wrong word

eroOP•2w ago

ah, i suppose Proj1

public interface IRequest<in TVisitor>
{
    void Visit(TVisitor visitor);
}

public sealed class RequestMessage<TRequestPayload>(TRequestPayload payload)
{
    public TRequestPayload Payload { get; } = payload;
}

public abstract class ServerBase<TRequestPayload>
{
    protected abstract void HandleMessage(RequestMessage<TRequestPayload> message);
}

public interface IRequest<in TVisitor>
{
    void Visit(TVisitor visitor);
}

public sealed class RequestMessage<TRequestPayload>(TRequestPayload payload)
{
    public TRequestPayload Payload { get; } = payload;
}

public abstract class ServerBase<TRequestPayload>
{
    protected abstract void HandleMessage(RequestMessage<TRequestPayload> message);
}

Proj2

[JsonDerivedType(typeof(GetFooRequest), nameof(GetFooRequest))]
public interface IFooRequest : IRequest<IFooRequestVisitor>;

public sealed class GetFooRequest : IFooRequest
{
    public void Visit(IFooRequestVisitor visitor)
    {
        visitor.Accept(this);
    }
}

public interface IFooRequestVisitor
{
    void Accept(GetFooRequest request);
}

public sealed class FooServer : ServerBase<IFooRequest>, IFooRequestVisitor
{
    protected override void HandleMessage(RequestMessage<IFooRequest> message)
    {
        message.Payload.Visit(this);
    }

    public void Accept(GetFooRequest request)
    {
        // ...
    }
}

[JsonDerivedType(typeof(GetFooRequest), nameof(GetFooRequest))]
public interface IFooRequest : IRequest<IFooRequestVisitor>;

public sealed class GetFooRequest : IFooRequest
{
    public void Visit(IFooRequestVisitor visitor)
    {
        visitor.Accept(this);
    }
}

public interface IFooRequestVisitor
{
    void Accept(GetFooRequest request);
}

public sealed class FooServer : ServerBase<IFooRequest>, IFooRequestVisitor
{
    protected override void HandleMessage(RequestMessage<IFooRequest> message)
    {
        message.Payload.Visit(this);
    }

    public void Accept(GetFooRequest request)
    {
        // ...
    }
}

i have this for now, will deal with any package incompatibilities later the only thing that bothers me here is that the visitor implementation is public

canton7•2w ago

You can impl it explicitly, or on an inner class I don't think you want in TVisitor? No need to be contravariant

eroOP•2w ago

with variance i almost always add it when it's possible

this_is_pain•2w ago

the way you are explaining this makes it a problem of the physical layer, not of the de/serializer 🤔 like sending length+data or bom+data+eom instead of raw bytes

eroOP•2w ago

hm, annoying issue: https://github.com/dotnet/runtime/issues/81840 seems like i need to length-prefix all of my sent data and read as an array of bytes instead

canton7•2w ago

Yeah, that's expected when using any streaming transport. Tcp and serial are the same The word you're looking for is "framing". Normally you'd have some sort of "start of message" marker so that the receiver can re-synchronise if it becomes desynced, and a crc (although that's probably not necessary for named pipes)

eroOP•2w ago

i just do it like this

public static void Serialize<T>(Stream stream, T value, JsonSerializerContext context)
{
    byte[] bytes = JsonSerializer.SerializeToUtf8Bytes(value, typeof(T), context);

    byte[] rented = ArrayPool<byte>.Shared.Rent(sizeof(int));
    BinaryPrimitives.WriteInt32LittleEndian(rented, bytes.Length);

    stream.Write(rented, 0, sizeof(int));
    stream.Write(bytes, 0, bytes.Length);

    ArrayPool<byte>.Shared.Return(rented);
}

public static void Serialize<T>(Stream stream, T value, JsonSerializerContext context)
{
    byte[] bytes = JsonSerializer.SerializeToUtf8Bytes(value, typeof(T), context);

    byte[] rented = ArrayPool<byte>.Shared.Rent(sizeof(int));
    BinaryPrimitives.WriteInt32LittleEndian(rented, bytes.Length);

    stream.Write(rented, 0, sizeof(int));
    stream.Write(bytes, 0, bytes.Length);

    ArrayPool<byte>.Shared.Return(rented);
}

i'm decently far along now, need to work on the responses next, really not sure how i wanna do that there's also the problem of having to handle a close command as well as internal server errors, which should be kinda shared between all server impls

canton7•7d ago

You can put extra fields on your base RequestMessage/ResponseMessage

eroOP•7d ago

and then?

canton7•7d ago

And then use them to encode errors / close messages / etc

eroOP•7d ago

oh i thought you meant for the length prefixing stuff still

canton7•7d ago

For the length prefixing - looks fine. As long as you don't need to resync if the receiver misses some bytes, or has to drop some bytes. You can probably get away with a uint16 if you care, and stackalloc will be a bit cheaper, but meh, it's fine

eroOP•7d ago

can't stackalloc, since streams on ns2.0 don't take spans

canton7•7d ago

Ah fair enough One way to model server errors / internal messages would be to do the same polymorphism as you do for payload but on the whole message. I'm on mobile so typing code is hard, but you have a base ResponseMessage class, and polymorphic subclasses for 1) a user payload (as you do now), 2) an error, 3) a close message, etc Or you can just stick extra fields on your current ResponseMessage and have an invariant that only one can be populated at a time

eroOP•7d ago

hm, i still need some help figuring out lots of parts on the client side as well as the responses, and also how i should implement the server polling for new data for the client, i'm envisioning something like this

public sealed class MonoClient : IpcClient<IMonoRequest, IMonoResponse>
{
    // Result<T> is my own type
    public Result<GetMonoImageResponse> GetMonoImage(string imageName)
    {
        base.SendRequest(
            new IpcRequestMessage<IMonoRequest>(
                new GetMonoImageRequest(imageName)));

        // receive? visitor pattern here? how?
    }
}

public sealed class MonoClient : IpcClient<IMonoRequest, IMonoResponse>
{
    // Result<T> is my own type
    public Result<GetMonoImageResponse> GetMonoImage(string imageName)
    {
        base.SendRequest(
            new IpcRequestMessage<IMonoRequest>(
                new GetMonoImageRequest(imageName)));

        // receive? visitor pattern here? how?
    }
}

i've changed some of the names, this is what i actually have in my code. so i can then just call client.GetMonoImage("Assembly-CSharp") i'm not sure if the server should be in charge of polling for new requests itself, or if that should be handled outside of the server;

public abstract class IpcServer<TRequestPayloadBase, TResponsePayloadBase> : IDisposable
{
    public void Start()
    {
        _pipe.WaitForConnection();

        while (true) // while (_pipe.IsConnected)?
        {
            var message = IpcSerializer.Deserialize<IpcRequestMessage<TRequestPayloadBase>>(_pipe, SerializerContext);
            if (message is null)
            {
                // ?
                continue;
            }

            IpcResponseMessage<TResponsePayloadBase> response;
            try
            {
                response = HandleMessage(message);
            }
            catch (Exception ex)
            {
                // ?
            }

            IpcSerializer.Serialize(_pipe, response, SerializerContext);
        }
    }

    protected abstract IpcResponseMessage<TResponsePayloadBase> HandleMessage(IpcRequestMessage<TRequestPayloadBase> request);
}

public abstract class IpcServer<TRequestPayloadBase, TResponsePayloadBase> : IDisposable
{
    public void Start()
    {
        _pipe.WaitForConnection();

        while (true) // while (_pipe.IsConnected)?
        {
            var message = IpcSerializer.Deserialize<IpcRequestMessage<TRequestPayloadBase>>(_pipe, SerializerContext);
            if (message is null)
            {
                // ?
                continue;
            }

            IpcResponseMessage<TResponsePayloadBase> response;
            try
            {
                response = HandleMessage(message);
            }
            catch (Exception ex)
            {
                // ?
            }

            IpcSerializer.Serialize(_pipe, response, SerializerContext);
        }
    }

    protected abstract IpcResponseMessage<TResponsePayloadBase> HandleMessage(IpcRequestMessage<TRequestPayloadBase> request);
}

basically should the loop be in the server here or should i handle this polling in my app code?

using var server = new MonoServer();
while (true)
{
    server.ProcessNextMessage();
}

using var server = new MonoServer();
while (true)
{
    server.ProcessNextMessage();
}

then there's the obvious question about how to design the response message. is something like this just enough?

public sealed record IpcResponseMessage<TPayload>(
    TPayload? Payload = default,
    string? Error = default);

public sealed record IpcResponseMessage<TPayload>(
    TPayload? Payload = default,
    string? Error = default);

and perhaps for later; i'd like to add some logging via some ILogger interface. i wanna implement console logging, file logging, and debug logging (Debug.WriteLine)

this_is_pain•7d ago

(just as an indication, how many messages do you expect that will travel on this channel? like 100/s or 1/s or 1/h)

eroOP•7d ago

it really depends on the user. some might only need 3 or 4 MonoFields from 1 MonoClass in 1 MonoImage others may request 100 fields from a total of 15 classes in 2 images the important bit is that the user is expected to request all of the data they need once and only once

this_is_pain•7d ago

but aren't these commands periodical? ah, that answers my question

eroOP•7d ago

so at (their) app startup, request all of the things they need (can be hundreds of requests sent as quickly as possible), and then the server might lay dormant. they can continue requesting more data, but it's not the expected use case i might expand the library to support generic memory reading from the target process, but i think a memory mapped file may be more useful there

this_is_pain•7d ago

but so then why do speak about polling? you mean in a general sense, not periodical polling

eroOP•7d ago

well the server still needs to read each new request (of those possible hundreds at app startup) in a row so it needs to wait for a new request to be sent in a loop that's my definition of polling for new messages

this_is_pain•7d ago

(can be hundreds of requests sent as quickly as possible)

wouldn't you consider batching requests then? (i mean sure ok first the whole system needs to work, but still)

eroOP•7d ago

Possibly, yeah The idea was kinda

var asmCs = client.GetMonoImage("Assembly-CSharp");

var player = client.GetClass(asmCs.Address, "" /* root namespace */, "Player");

var hp = client.GetField(player.Address, "_hp");

Console.WriteLine(hp.OffsetInternal);

var asmCs = client.GetMonoImage("Assembly-CSharp");

var player = client.GetClass(asmCs.Address, "" /* root namespace */, "Player");

var hp = client.GetField(player.Address, "_hp");

Console.WriteLine(hp.OffsetInternal);

Beyond that is a bit convoluted and idk if explaining it makes sense. Users will want to read the value of different class' fields. For that, they need to build a path of offsets (from the start of the class instance to the individual fields) that needs to be entirely dereferenced to actually read the value. Say the game contains something like this

class GameManager
{
  static GameManager _instance;
  Player _player;
}

class Player
{
  int _hp;
}

class GameManager
{
  static GameManager _instance;
  Player _player;
}

class Player
{
  int _hp;
}

The user first needs the static data chunk address of GameManager, add the offset of GameManager._instance to it, dereference, add the offset of GameManager._player to the result, dereference that, and finally add the offset of Player._hp to read the value at the result. So something like this;

GameManager._instance._player._id
*(int*)(*(*(0x12345678 + 0x0) + 0x10) + 0x3C)

GameManager._instance._player._id
*(int*)(*(*(0x12345678 + 0x0) + 0x10) + 0x3C)

So once they have all the addresses and offsets, they "build" their path and want to "watch" the value at the end of that path. I can do this via ReadProcessMemory calls, but each offset (each dereference) is one RPM call That's where I thought mem mapped files would work well (just continuously make the server update the value)

this_is_pain•7d ago

so imagine player dies and user reloads save, to re-bind hp field the program would have to re-calculate at least the last offset, i guess

eroOP•7d ago

No, the offsets always stay the same

canton7•7d ago

Neither, IMO. You want an additional layer which reading the incoming stream, decoding responses, matching them to pending requests, and completing the relevant TCSs (see my messages about correlation tokens from yesterday) (that layer sits below your IpcClient/IpcServer of course)

eroOP•7d ago

What's lower than an abstract base class?

canton7•7d ago

Uh... We architect things in layers. abstract is irrelevant to that

this_is_pain•7d ago

the answer was: another abstract class

canton7•7d ago

Higher layers call down into lower layers. Higher layers deal with more abstract stuff, lower layers deal with more nitty-gritty detail Hah! I wouldn't here, though. For clarity. Composition over inheritance etc

eroOP•7d ago

idk how you'd do it then

canton7•7d ago

Have a class which wraps the named pipe. It takes a message in (and sends it over the pipe), and it had a constantly running task which reads messages from the pipe

eroOP•7d ago

That's what the abstract ipcserver class does Exactly that, wraps the pipe

canton7•7d ago

On the client, you wrap that in a class which takes requests, sends them, gets responses, matches the responses up to pending requests Then the IpcClient wraps that Don't just bung everything in one class. That's basic architecture. Identify what the different responsibilities are, and build your abstractions

eroOP•7d ago

I really don't follow at all

canton7•7d ago

What's confusing you?

eroOP•7d ago

i'm just not really that sure what you're suggesting in the first place i guess. it doesn't make sense to me to add an additional wrapper around just the pipe that does the "polling" and the de/serialization. my base server is already that wrapper. the actual child servers then use the wrapper in the form of a base class. adding another class here just seems like overcomplicating it. i'd rather address the other stuff i brought up

canton7•6d ago

Which bit are you still struggling with? I thought I addressed your latest round

eroOP•6d ago

this basically summed it up but if you have anything to say to this, I'm open to hear your reasoning in a bit more detail, maybe with an example so I have something concrete in front of me. sorry if it just ends up in you repeating yourself, I'm just trying to understand why you think it's a better idea to further decouple the pipe and the server like that

canton7•6d ago

That's just basic separation of concerns. They can get quite large: in my current project, the bit thay handles sequencing, aborting, and timing out messages is 350 lines; the thing that handles framing / deftaming messages is 160 lines; the thing that handles reading and writing byes to tcp is 270 lines; and the thing that handles deciding what to connect to, reconnections, disconnection, connection checks, etc, is 300 lines (and that's just a client: the server is an embedded device, so that's in C) My answer to thay was just "yes, both sides need to have a loop which just reads the pipe for new received bytes". Both the server and the client Were there any other questions in there? "lots of parts of the client side" is kinda hard to answer You probably want to support having multiple messages in flight at once, so the server can take a while to respond to a message if it wants. For that you need one component which reads bytes from the pipe and assembles them back into messages, then a component which takes those messages, matches them up to pending request, and notifies the requester that it's got a response

eroOP•6d ago

Well I wanted to know how I could go about the client implementation with the way I was envisioning it in this message https://discord.com/channels/143867839282020352/1339224833359347822/1346481941335113749

MODiX•6d ago

ero

for the client, i'm envisioning something like this

public sealed class MonoClient : IpcClient<IMonoRequest, IMonoResponse>
{
    // Result<T> is my own type
    public Result<GetMonoImageResponse> GetMonoImage(string imageName)
    {
        base.SendRequest(
            new IpcRequestMessage<IMonoRequest>(
                new GetMonoImageRequest(imageName)));

        // receive? visitor pattern here? how?
    }
}

public sealed class MonoClient : IpcClient<IMonoRequest, IMonoResponse>
{
    // Result<T> is my own type
    public Result<GetMonoImageResponse> GetMonoImage(string imageName)
    {
        base.SendRequest(
            new IpcRequestMessage<IMonoRequest>(
                new GetMonoImageRequest(imageName)));

        // receive? visitor pattern here? how?
    }
}

i've changed some of the names, this is what i actually have in my code.

Quoted by

<@542772576905199626> from #Named Pipe IPC: Protocols & Status Codes (click here)

React with ❌ to remove this embed.

eroOP•6d ago

Or if you think it's perhaps a bad idea to make the client take raw arguments instead of entire RequestMessages And whether taking entire RequestMessages would allow for other patterns once again

canton7•6d ago

That seems fine. You probably want somewhere which links together a request message type and a response message type. Thay might be in the call to SendRequest itself, or you could add the type of the response to the type of the request (as a generic type param), or have some other type which links them together. Either way, that means your SendRequest method can return the actual type of the response, and the user doesn't need to worry I'd just send the payload, not the whole request message The bits of the request message other than the payload are for use by your library, not for the user to use So, this should be a request message payload type with a response message payload type

eroOP•6d ago

That sounds reasonable, what I would like to know is what that could enable me to do in terms of patternizing (?? idk what word I'm looking for here) the exchange of the request and response data Coupling them together and such

canton7•6d ago

Eg in my current project I do GetFooResponsePayload response = await TransmitAsync(Command.GetFoo, new GetFooRequestPayload(...)). Where Command.GetFoo is a static new Command<GetFooRequestPayload, GetFooResponsePayload>() (actually it contains some other data as well, but that's not relevant) In terms of coupling the types, see above, or you could define your GetFooRequestPayload as a RequestPayloadBase<GetFooResponseMessage> and tie them together in the type system that way In terms of pairing a response message back to the request which triggered it, search this thread for "correlation token" Or you could just call SendAsync<FooRequestPayload, FooReaponsePayload>(new FooRequestPayload(...)) and tie them together at the call site (I've done all of those at one point or another)

eroOP•8h ago

i can't figure it out idk? this can surely not be it

public interface IRequest<TRequestVisitor, TResponse, TResponseVisitor>
    where TResponse : IResponse<TResponseVisitor>
{
    IpcResponseMessage<TResponse> Visit(TRequestVisitor visitor);
}

public interface IMonoRequest<TResponse> : IRequest<IMonoRequestVisitor, TResponse, IMonoResponseVisitor>
    where TResponse : IMonoResponse;

public interface IRequest<TRequestVisitor, TResponse, TResponseVisitor>
    where TResponse : IResponse<TResponseVisitor>
{
    IpcResponseMessage<TResponse> Visit(TRequestVisitor visitor);
}

public interface IMonoRequest<TResponse> : IRequest<IMonoRequestVisitor, TResponse, IMonoResponseVisitor>
    where TResponse : IMonoResponse;

can't even use it in the IpcServer base class then anymore because i have no base payload my brain is fried i don't get it @canton7 I guess yeah i can't figure it out i really tried I beg you @canton7 get me out of this hell hole

Gaming

Programming

Named Pipe IPC: Protocols & Status Codes

Did you find this page helpful?