536
Java (lemmy.world)
you are viewing a single comment's thread
view the rest of the comments
[-] unawareallium@sh.itjust.works 3 points 1 year ago

You can think of a double as having a fixed precision, but, in contrast to an integer, this precision can be moved over the decimal point depending on the value you want to represent. Therefore, despite representing floating-point numbers, a double still has discrete steps determined by its binary representation of 64 bits. If the value of a double gets larger, it reaches a point where the smallest difference between two subsequent doubles is greater than one. For float (32 bit), you reach this point at 16777216. The next larger number to be represented as a float is 16777218 (i.e., +2).

Here is a nice online tool that demonstrates this (and contains much more information on the encoding of floating-point numbers): https://www.h-schmidt.net/FloatConverter/IEEE754.html

[-] karlthemailman@sh.itjust.works 1 points 1 year ago

I agree with all that. But I'm talking about exact integer values as mentioned in the parent.

I just think this has to be true: count(exact integers that can be represented by a N bit floating point variable) < count(exact integers that can be represented by an N bit int type variable)

this post was submitted on 05 Aug 2023
536 points (92.4% liked)

Programmer Humor

19187 readers
1139 users here now

Welcome to Programmer Humor!

This is a place where you can post jokes, memes, humor, etc. related to programming!

For sharing awful code theres also Programming Horror.

Rules

founded 1 year ago
MODERATORS