❔ Runtime Endianness for Reading Structured Binary Data
Is there a standard way to deal with reading structured binary data in from a stream where the endianness is known only at runtime? In other words, it would be great if there was something with the same API as the
BinaryReader but where the endianness for the reader could be set.
I'm pretty new to C# so I didn't know if there was a idiomatic way to do this. In go I have used the binary.Read (https://pkg.go.dev/encoding/binary#Read) API to allow for reading in primitive types while specifying the endianness. Similarly, I have used the Java ByteBuffer type (https://docs.oracle.com/javase/8/docs/api/java/nio/ByteBuffer.html) to accomplish this on the JVM.
From my research, I have discovered (correct me if I'm wrong) that everything in dotnet is little endian. I have also found the BinaryPrimatives class which offers different methods for reading/writing binary primitives with different endiannesses. However, using this would require conditional statements wherever a read is performed. I wasn't sure if there was an extant class that provides a sleeker API for this. I know I could wrap it myself, but didn't want to reinvent the wheel if it was just something I missed18 Replies
Nothing built-in
I'm not sure whether there's a NuGet for it, but writing it yourself shouldn't be too difficult
Ah, gotcha. Okay, well thanks for the info! Figured I'd ask before writing code haha
BitConverter.IsLittleEndian (https://learn.microsoft.com/en-us/dotnet/api/system.bitconverter.islittleendian?view=net-7.0) is a runtime constant, so basically the if-check of something like
will be reduced to the respective method calloh interesting, i would have solved it with an interface and 2 implementations
actually yeah i don't think that's what they want lol
The endianness of the stream I'm reading is not necessarily that of the host system. It's encoded in a file at a specific location, I read that to determine the endianness of the file, then have to adjust the loads accordingly.
same here tbh, its less bloat in regards of writing both variants - tho i usually use SGs for such work
well, then the 2 classes + 1 interface approach would be the best
Thanks, y'all!
are u forced to use a specifc way for the binary (de-)serialization? if not there are quite some libs out there to not re-invent the wheel
There's no restriction on the method I used to go from binary on disk -> object, so any libraries that could help would be appreciated. I'm new to the C# ecosystem, so not familiar with the usual suspects when it comes to this
MessagePack is a quite known one: https://github.com/MessagePack-CSharp/MessagePack-CSharp
GitHub
GitHub - MessagePack-CSharp/MessagePack-CSharp: Extremely Fast Mess...
Extremely Fast MessagePack Serializer for C#(.NET, .NET Core, Unity, Xamarin). / msgpack.org[C#] - GitHub - MessagePack-CSharp/MessagePack-CSharp: Extremely Fast MessagePack Serializer for C#(.NET,...
there is also google's protobuf
https://github.com/Cysharp/MemoryPack is binary stuff too
GitHub
GitHub - Cysharp/MemoryPack: Zero encoding extreme performance bina...
Zero encoding extreme performance binary serializer for C# and Unity. - GitHub - Cysharp/MemoryPack: Zero encoding extreme performance binary serializer for C# and Unity.
dunno which will fit ur needs, maybe none of these, but it gives ya starting point to look around for options or to decide to go for something self-written
its also a matter what ur use case is. no need to use something high performant but complex or something selfwritten if its in a cold path
Awesome, thank you so much for that! I will look through these and see how I can fit it into my goals
one last hint: if u need high performance, use source generator based solutions
i don't see how source gen would help here
they said they only know the endianness at runtime
so sg doesn't really make sense as a solution
that was apart from the endianness thingy, and u can let these libs generate both versions, for big and little endianness iirc
Was this issue resolved? If so, run
/close - otherwise I will mark this as stale and this post will be archived until there is new activity.