❔ Why does it do this?
I'm learning C# and this makes no sense the answer is 3.3 and that's it why does it try to exact it and make it wrong still?
7 Replies
Floating point rounding errors
That's just how floating points work. Not every floating point number can be represented in binary, so what we often get is the best approximation
oh ok thank you!
there is a nice tool here that you can use to experiment and see why this happens https://www.h-schmidt.net/FloatConverter/IEEE754.html
🎵 pigeon hole principle 🎵
For true precision, you can use the
decimal type.
https://learn.microsoft.com/en-us/dotnet/api/system.decimal?view=net-7.0#remarksDecimal Struct (System)
Represents a decimal floating-point number.
decimal stil has limitations
it's just much better in the precision department than float or doubleWas this issue resolved? If so, run
/close - otherwise I will mark this as stale and this post will be archived until there is new activity.