What Egor is trying to say is that computers do not calculate exact answers most of the time.
Texas Instruments deals primarily in the micro-controllers so I wouldn't expect the usual x86-64 processor inside that device of yours. That means TI can do a whole lot of things in their own way. They may make own decisions on how to handle small values, rounding, how to handle complex mathematical operations etc...
Computers use at least 32 bit floating point numbers these days. This page gives precision (number of bits before e^-16
that are correct in the machine representation). For 32 bits that value is 24
. I couldn't find much info on the calculator, except the wiki page, which says its precision is 14
. More than half-float less than float, not defined in that IEEE standard.
That sqrt
over there is a nasty function. Computing its value requires quite a lot of computations. Lots of steps means lots of arithmetic errors, the lower precision the further from true value it gets. This also depends on the exact algorithm chosen in the sqrt
function. You could check whether the math.sqrt(4^2)
returns what it should return and whether math.sqrt(4^2))/(2*4)
return exactly half of that.
Battling with numerical errors in computations is an entire discipline on its own and recipes differ depending on what equation you are tackling.
There's this post considering the quadratic equations.
Or maybe in your case you'll be happier with just discarding all but few numbers after the decimal point in the final answer.