Round behaves differently than MRI/JRuby

[ntkme]

MRI

irb(main):001> a = 0.8241000000000004.round(10)
=> 0.8241
irb(main):002> b = 0.8241000000000002.round(10)
=> 0.8241
irb(main):003> a == b
=> true

TruffleRuby

irb(main):001:0> a = 0.8241000000000004.round(10)
=> 0.8241
irb(main):002:0> b = 0.8241000000000002.round(10)
=> 0.8240999999999999
irb(main):003:0> a == b
=> false

[ntkme]

#3360 might be related

[andrykonchin]

Thank you for the report, we'll look into it.

[eregon]

Something interesting about these 2 float values is they seem to be exact:

irb(main):013:0> "%.100f" % 0.8241000000000002
=> "0.8241000000000002000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
irb(main):014:0> "%.100f" % 0.8241000000000004
=> "0.8241000000000004000000000000000000000000000000000000000000000000000000000000000000000000000000000000"

So at first sight this does look like a bug in rounding.
OTOH, floats are always imprecise and the process of rounding itself can lose precision (otherwise it would need very large amounts of memory & time in some edge cases).

[ntkme]

floats are always imprecise and the process of rounding itself can lose precision

I understand and that's why I only used "behave differently" in the title. What's interesting to me is that JRuby also runs on JVM, but it managed to behave the same way as MRI.

Also, the reason for rounding in the first place is an attempt to reduce the impact of float precision error in sass-embedded gem. In Sass language, we define "fuzzy equality" as:

Two doubles are said to be fuzzy equal to one another if either:

• They are equal according to the compareQuietEqual predicate as defined by IEEE 754 2019, §5.11.
• They are both finite numbers and the mathematical numbers they represent produce the same value when rounded to the nearest 1e⁻¹¹ (with ties away from zero).

This "fuzzy equality" is necessary because we have Sass implementation across Dart, JS, Ruby, C++, Java and many other languages, and different language implementations often lead to different float precision for math operations.

Specifically, Sass recently added support for CSS Color Level 4 which introduced the conversion of colors between color spaces. This involves lots of mathematic operation. What happened is that TruffleRuby generally had higher precision errors than MRI/JRuby, comparing to the Dart reference implementation. This rounding behavior difference is just the most obvious case that would occasionally fail the fuzzy equality test immediately.

[ntkme]

This can be worked around as:

irb(main):001:0> (0.8241000000000004 * 10**10).round.fdiv(10**10)
=> 0.8241
irb(main):002:0> (0.8241000000000002 * 10**10).round.fdiv(10**10)
=> 0.8241