C#•12mo ago

❔ Read bits from a byte[] array

Hi, I was looking for a way to read bits in a byte[] array and advance the position, and I came across the BitStream library (https://github.com/Sewer56/Sewer56.BitStream) because we can't do it natively in C#. The problem is that I've noticed that I'm not getting the expected behavior in my code At first, I need to read an int32 in my byte stream, so I did :

var stream = new ArrayByteStream(buffer);
var bitStream = new BitStream<ArrayByteStream>(stream);
var header = bitStream.Read<int>();
Console.WriteLine(header);

var stream = new ArrayByteStream(buffer);
var bitStream = new BitStream<ArrayByteStream>(stream);
var header = bitStream.Read<int>();
Console.WriteLine(header);

And in output I get :

Normally I'd get something shorter like this:

I don't want to write a code that will only retrieve the first two characters because I don't necessarily know the size Do you know why the BitStream library reacts like this? When I use MemoryStream + BinaryReader it works perfectly except that I get stuck when I have to read bits in a stream... Thanks a lot 🙏

50 Replies

ero•12mo ago

So you do Read<int>, but expect it to read a byte and skip the next 3 bytes?

Valwex•12mo ago

Thanks for your answer In the library, using Read<int>() means that I want to read 32 bits (4 bytes), and advance the position by 4 bytes.

nukleer bomb•12mo ago

because we can't do it natively in C#

Absolutely wrong. Just to be clear: you want to read int32 from byte[]?

ero•12mo ago

And is that not the exact behavior you're seeing?

Valwex•12mo ago

Yes, I know that we can read an int32 natively EXCEPT that in the rest of my code I have to read bits and we can't do that natively

ero•12mo ago

It's difficult to understand your goal honestly

nukleer bomb•12mo ago

bool GetBit(int value, int bit) => (value & (1 << bit)) != 0;

bool GetBit(int value, int bit) => (value & (1 << bit)) != 0;

Is this what you need?

Valwex•12mo ago

Basically, I have a UDP client that must read packets sent to me by a UDP server In the stream I receive, I have to read both integers and bits In C#, we can only read bytes in a stream

ffmpeg -i me -f null -•12mo ago

there's also BitArray but i guess he knows

Valwex•12mo ago

Basically I have this:

public static void Parse(byte[] buffer)
{
  var stream = new ArrayByteStream(buffer);
        
  var bitStream = new BitStream<ArrayByteStream>(stream);

  // Reads 4 bytes and advances the position by 4 bytes
  var header = bitstream.ReadInt32();
        
  // Read 3 bits and advance position by 3 bits
  var subchanIndex = bitstream.ReadBits(3);
}

public static void Parse(byte[] buffer)
{
  var stream = new ArrayByteStream(buffer);
        
  var bitStream = new BitStream<ArrayByteStream>(stream);

  // Reads 4 bytes and advances the position by 4 bytes
  var header = bitstream.ReadInt32();
        
  // Read 3 bits and advance position by 3 bits
  var subchanIndex = bitstream.ReadBits(3);
}

Except that the values I get are much larger than if I use MemoryStream + StreamReader

ffmpeg -i me -f null -•12mo ago

larger? ah ok the 671088640

Valwex•12mo ago

Yep

nukleer bomb•12mo ago

Well, this works as expected. Read<int>() reads one 32-bit integer and advances position by 4 bytes. And values around 600M are reasonable for 32bit int. If you want to read a single byte, then Read<byte>()

Valwex•12mo ago

Okay, but why get a different value in the console? If I use MemoryStream + BinaryReader or something else, I'll get the same values, but smaller

nukleer bomb•12mo ago

ReadByte() from *Stream reads exactly one 8bit integer ('byte'), but returns it as a 32bit integer (int) https://learn.microsoft.com/dotnet/api/system.io.stream.readbyte

Valwex•12mo ago

I think it's a cast problem, yes, because in my code there are conditions that check whether the header variable corresponds to something and I never get to that condition, whereas if I use MemoryStream + BinaryReader it works

ffmpeg -i me -f null -•12mo ago

BinaryReader.Read returns a byte eh, late but are you trying to make a integer from an arbitrary number of bits?

nukleer bomb•12mo ago

So use BitStream.Read<byte>() to get the same behaviour as MemoryStream.ReadByte()

Valwex•12mo ago

Yes, I get the same behavior, but from the moment I start reading 4 bytes I don't get the same values

ero•12mo ago

yes, because reading 4 bytes is not the same as reading 1 byte

Valwex•12mo ago

// I get two different results here
// I would like to obtain the same result as header2 in header 1
var header1 = bitStream.Read32(32);
var header2 = BitConverterToInt32(binReader.ReadBytes(4));

// I get two different results here
// I would like to obtain the same result as header2 in header 1
var header1 = bitStream.Read32(32);
var header2 = BitConverterToInt32(binReader.ReadBytes(4));

ffmpeg -i me -f null -•12mo ago

(wouldn't be better using uint instead of int? btw)

Valwex•12mo ago

I need to read negative values in the header variable because basically the packet id can be -1, -3 etc

ffmpeg -i me -f null -•12mo ago

weird but ok i guess

Valwex•12mo ago

To keep things simple, the server will write a int32 at the beginning of the packet, which I must read to retrieve the value

ffmpeg -i me -f null -•12mo ago

did you check if the bitStream endianness == binReader endianness?

MODiX•12mo ago

ero

REPL Result: Success

#nuget Sewer56.BitStream

using Sewer56.BitStream;
using Sewer56.BitStream.ByteStreams;

byte[] bytes = { 0x12, 0x34, 0x56, 0x78 };

ArrayByteStream abs = new(bytes);
BitStream<ArrayByteStream> bs = new(abs);

MemoryStream ms = new(bytes);
BinaryReader br = new(ms);

uint u1 = bs.Read32(32);
uint u2 = BitConverter.ToUInt32(br.ReadBytes(4));

($"{u1:X}", $"{u2:X}")

#nuget Sewer56.BitStream

using Sewer56.BitStream;
using Sewer56.BitStream.ByteStreams;

byte[] bytes = { 0x12, 0x34, 0x56, 0x78 };

ArrayByteStream abs = new(bytes);
BitStream<ArrayByteStream> bs = new(abs);

MemoryStream ms = new(bytes);
BinaryReader br = new(ms);

uint u1 = bs.Read32(32);
uint u2 = BitConverter.ToUInt32(br.ReadBytes(4));

($"{u1:X}", $"{u2:X}")

Result: ValueTuple<string, string>

{
  "item1": "12345678",
  "item2": "78563412"
}

{
  "item1": "12345678",
  "item2": "78563412"
}

Compile: 987.689ms | Execution: 73.854ms | React with ❌ to remove this embed.

ero•12mo ago

Read32 from BitStream returns a uint

Valwex•12mo ago

BitStream reads big endian data https://github.com/Sewer56/Sewer56.BitStream/issues/2

ero•12mo ago

i don't think it does

MODiX•12mo ago

ero

REPL Result: Failure

byte[] bytes = { 0x12, 0x34, 0x56, 0x78 };

var bigEndian = BinaryPrimitives.ReadInt32BigEndian(bytes);
var littleEndian = BinaryPrimitives.ReadInt32LittleEndian(bytes);

($"{bigEndian:X}", $"{littleEndian:X}")

byte[] bytes = { 0x12, 0x34, 0x56, 0x78 };

var bigEndian = BinaryPrimitives.ReadInt32BigEndian(bytes);
var littleEndian = BinaryPrimitives.ReadInt32LittleEndian(bytes);

($"{bigEndian:X}", $"{littleEndian:X}")

Exception: CompilationErrorException

- The name 'BinaryPrimitives' does not exist in the current context
- The name 'BinaryPrimitives' does not exist in the current context

- The name 'BinaryPrimitives' does not exist in the current context
- The name 'BinaryPrimitives' does not exist in the current context

Compile: 502.803ms | Execution: 0.000ms | React with ❌ to remove this embed.

ero•12mo ago

MODiX•12mo ago

ero

REPL Result: Success

using System.Buffers.Binary;

byte[] bytes = { 0x12, 0x34, 0x56, 0x78 };

var bigEndian = BinaryPrimitives.ReadInt32BigEndian(bytes);
var littleEndian = BinaryPrimitives.ReadInt32LittleEndian(bytes);

($"{bigEndian:X}", $"{littleEndian:X}")

using System.Buffers.Binary;

byte[] bytes = { 0x12, 0x34, 0x56, 0x78 };

var bigEndian = BinaryPrimitives.ReadInt32BigEndian(bytes);
var littleEndian = BinaryPrimitives.ReadInt32LittleEndian(bytes);

($"{bigEndian:X}", $"{littleEndian:X}")

Result: ValueTuple<string, string>

{
  "item1": "12345678",
  "item2": "78563412"
}

{
  "item1": "12345678",
  "item2": "78563412"
}

Compile: 567.905ms | Execution: 68.540ms | React with ❌ to remove this embed.

ero•12mo ago

guess it does

Valwex•12mo ago

I get 0x00000035 when using MemoryStream + BinaryReader and 0x35000000 when using BitStream

ffmpeg -i me -f null -•12mo ago

which is pretty self explanatory

Valwex•12mo ago

Yes, except that there's no lib capable of reading a byte stream in little endian :/

ero•12mo ago

yeah i mean i laid out very clearly why that is and i also provided some code that reads a byte buffer as little endian

Valwex•12mo ago

Yes, it works, but I can't read several values in my byte stream

ero•12mo ago

hm?

Valwex•12mo ago

For example, if I then want to read 3 bits from my byte stream, the ReadInt32BigEndian method won't advance the stream by 4 bytes, do you understand?

ero•12mo ago

i mean i would read an int and then do & 0b111 on the result

Valwex•12mo ago

Do you have an example, please? Because I'm lost :x

ffmpeg -i me -f null -•12mo ago

you can take what the stream is returning, so at least a byte then wrap it to consume the bits i guess he's saying that obviously it means writing your own bit stream

MODiX•12mo ago

ero

REPL Result: Success

int i = 0b1010_1111;
(i, i & 0b111)

int i = 0b1010_1111;
(i, i & 0b111)

Result: ValueTuple<int, int>

{
  "item1": 175,
  "item2": 7
}

{
  "item1": 175,
  "item2": 7
}

Compile: 468.504ms | Execution: 41.094ms | React with ❌ to remove this embed.

ero•12mo ago

i'm pretty sure there are way bigger underlying issues that we can't really help with without way more detail

ffmpeg -i me -f null -•12mo ago

more issues = more code = more fun right?

nukleer bomb•12mo ago

It's no problem to swap it manually after reading

var bytes = bitStream.ReadBytes(4);
if(BitConverter.IsLittleEndian)
   bytes.Reverse();
var header2 = BitConverter.ToInt32(bytes);

var bytes = bitStream.ReadBytes(4);
if(BitConverter.IsLittleEndian)
   bytes.Reverse();
var header2 = BitConverter.ToInt32(bytes);

Valwex•12mo ago

Yup

Accord•12mo ago

Was this issue resolved? If so, run /close - otherwise I will mark this as stale and this post will be archived until there is new activity.

Gaming

Programming

❔ Read bits from a byte[] array