0

I have a string which I want to convert to a decimal.
eg.

tax_rate = "0.07"
tax_rate.to_d

In most cases it converts to 0.07 but sometimes it converts it to 0.07000000000000001. Why?

If it helps, when I look at the log of when I insert this value into the DB, this is the relevant part:

["tax_rate", #<BigDecimal:7f9b6221d7c0,'0.7000000000 000001E-1',18(45)>]

Hopefully that makes sense to someone.

Disclaimer:
I have a feeling someone is going to ask why I'm doing this.
I've created a simple settings model, where a user can update some global settings.
The Setting model has name, value and var_type columns.
The value column is a string. I use a helper method to retrieve the value in the appropriate format depending on the value of the var_type column.

4

1 回答 1

1

我无法解释为什么，但有机会我可以告诉你如何避免在处理数字时遇到这种麻烦：使用有理数。

这是文档：http ://ruby-doc.org/core-1.9.3/Rational.html

如前所述，理性总是会给你你想要的确切数字，从而避免舍入错误。

从文档：

10.times.inject(0){|t,| t + 0.1}              #=> 0.9999999999999999
10.times.inject(0){|t,| t + Rational('0.1')}  #=> (1/1)

让我知道这是否可以解决您的问题。:)

于 2013-05-02T18:31:41.313 回答