Casio FX-5000F incorrect roots for quadratic?

[corrado33]

Hi all,
I figured with Dave's fondness of casios here, I'd be able to find an answer.

I have a bunch of old casio calculators (thanks Dave... ) My most recent addition is a Fx-5000F. I absolutely love it, so much more useful than newer calculators, and in a much smaller package. I especially love using the scientific formulae. That's what this question is about. I use the quadratic formula often, so I very often use the first formula on the calculator. Sure, I could solve it by hand, but using the formula on the calculator is much quicker. That's the point.

Anyway, I'm trying to solve a quadratic with A=1, B=0.15, C=-7.7E-13. Now, I realize C is quite small, but I've done many other calculations on the calculator which turned out accurate, even at those orders of magnitude.

When put into the first formula (Press 1, then FMLA), it gives roots of -0.15 (correct) and 4.5E-13(wrong). The actual other root is 5.1E-13. Any ideas?

Please don't berate me for using a calculator to do this. This is more of a question of WHY it's wrong, not that it IS wrong. It's easy to calculate this other ways obviously.

EDIT: I realize that it probably has something to do with the rounding in the calculator, but I'm not about to dig through the entire 200 page manual to find the specs on it.  In fact, if I change C to = -7.7E-12, the formula actually gives the correct answer. Gahh, maybe I will dig through the manual. I've read it once, I don't remember anything about accuracy in formulae though.

[IanB]

This is where you need to write the formula down on paper.

If you do that, you will see this calculation:

  -0.15 + sqrt(0.15^2 - 4*1*-7.7E-13)

You will notice a difference between two terms almost equal to each other, which will lead to round-off error. Work it out step by step and see what happens.

This is a classic kind of problem in numerical computation.

There is a way of re-ordering the calculations to avoid this problem with the quadratic formula. Google will probably find it for you.

[corrado33]

Ian,
I did work it out by hand and went "What the heck?" Most calculators will give you 0.15^2 when you do (0.15^2 - 4*-7.7E-13). Therefore, without another technique, using the quadratic equation with normal calculators won't give you the correct answer. I actually had to go to wolfram alpha to give me the correct answer. I believe it's supposed to be solved using completing the square.

I'm just flabbergasted. I've never had a calculator give me a blatantly wrong answer. (At least that I know of.) (Yes, the calculator doesn't "give" me an answer, but in this case it's not user error. I'm using it as intended.)

I THINK the reason for the rounding is because internally, the FX-5000F uses a 12 digit mantissa, and when you subtract the two numbers, the last digit (or two) is the 13th digit, and gets cut off/rounded. The calculator I was using to compare it to, a TI-83 Plus, uses a 14 digit mantissa, and therefore computes the correct answer. (However, if you use -7.7E-15, it will NOT give the right answer, because it too, rounds the last digit of the mantissa.)

I hope I haven't marked any student's answers wrong because of this. 

Note to self: Be careful when subtracting very small numbers from relatively large numbers (especially when the difference between the two is ~12-14 orders of magnitude.)

[corrado33]

I may as well spell it out now that I've (we've) figured it out.

Inside of the square root for the quadratic equation is b^2-4ac. 

B^2 = 0.0225
4ac = -3.08E-12 or .00000000000308

If you add (subtracting a negative = addition) B^2 and 4ac you get .02250000000308 or 2.250000000308E-2.

Now if you count, that's a 13 digit mantissa (the part before the "E", not including the decimal point.... actually, I'm not sure if the decimal point is included in the "counting" for the mantissa. If it is, then it's a 14 point mantissa.)

Therefore, the calculator internally either rounds or truncates it to 2.25000000031E-2 or 2.25000000030E-2 (assuming the decimal point isn't included in the mantissa counting.)

/or/

If the decimal point IS included in the mantissa counting it'd be 2.2500000003E-2

That rounding/truncation is the reason for the error. (And the reason that the TI-83 plus gets it right, with a 14 digit mantissa. Admittedly, the TI-83 plus is a MUCH newer calculator, so I'll let this one pass TI... Apparently the TI-89 uses a 16 digit mantissa!)

[IanB]

My venerable TI-58 (1977) had 12 10 display digits and 3 "guard" digits, making 15 13 in total. So even old calculators were capable of good accuracy (the TI-58/59 were aimed at engineers though, rather than students).

As to the round-off problem, this can be solved by observing that x1*x2 = c/a. Hence we can compute:

    x1 = [- 0.15^2 - sqrt(0.15^2 - 4 * -7.7e-13) ] / 2 = -0.15

    x2 = -7.7e-13 / -0.15 = 5.13E-12

This will work with even 8 digits of precision.

[corrado33]

My venerable TI-58 (1977) had 12 display digits and 3 "guard" digits, making 15 in total. So even old calculators were capable of good accuracy (the TI-58/59 were aimed at engineers though, rather than students).

As to the round-off problem, this can be solved by observing that x1*x2 = c/a. Hence we can compute:

    x1 = [- 0.15^2 - sqrt(0.15^2 - 4 * -7.7e-13) ] / 2 = -0.15

    x2 = -7.7e-13 / -0.15 = 5.13E-12

This will work with even 8 digits of precision.

Ah, that IS an easy way to do it. Even when trying to complete the square you still run into the same problem. Hmmm... I like these math puzzles where you have to think of different ways to do things. (Solve this problem using only X digits of precision.) Wonder if I can find a book that talks about this kind of stuff.

By the way, is the decimal point included in the counting for the mantissa digits? I mean, this number has to be stored in a register somewhere on the chip. So either the decimal point position has it's own register, or they represent it some other way all in one register. (Or a few registers, since it's so long, but you catch my drift.) Geeze, you'd need 10 bits of space to store a 12 digit number. I guess that's not too many now-a-day (even tiny micros have 12 bit ADCs sometimes), but I don't know how spectacular that was back in the 80s. (Or even if it was spectacular at all!)

[Len]

Hmmm... I like these math puzzles where you have to think of different ways to do things. (Solve this problem using only X digits of precision.) Wonder if I can find a book that talks about this kind of stuff.

You could get a textbook on numerical analysis if you want to kill your enthusiasm learn all the techniques of numeric computation.

By the way, is the decimal point included in the counting for the mantissa digits? I mean, this number has to be stored in a register somewhere on the chip. So either the decimal point position has it's own register, or they represent it some other way all in one register. (Or a few registers, since it's so long, but you catch my drift.)

The exponent specifies the position of the decimal point. For example, with -7.7e-13,  -7.7 is the mantissa (the significant digits) and -13 is the exponent that scales the mantissa by powers of 10 (which is the same as moving the decimal point).

Geeze, you'd need 10 bits of space to store a 12 digit number. I guess that's not too many now-a-day (even tiny micros have 12 bit ADCs sometimes), but I don't know how spectacular that was back in the 80s. (Or even if it was spectacular at all!)

In the 80's microprocessors only did integer arithmetic. Floating point was done either in software or by a separate co-processor chip. Floating point numbers are usually 32, 64 or 80 bits.

[IanB]

In the 80's microprocessors only did integer arithmetic. Floating point was done either in software or by a separate co-processor chip. Floating point numbers are usually 32, 64 or 80 bits.

However, hand-held calculators did not necessarily do binary arithmetic. It was more common to use some form of BCD for storage and computation.

[corrado33]

"Calculator forensics" is an area where there has been a lot of study lately. You can see the FX-5000F compared here: http://www.rskey.org/~mwsebastian/miscprj/models.htm

Yep, I've run across that site before. Very interesting stuff. I bet the majority of people don't realize that different calculators can give you different values for the same calculation. I mean, honestly you could use most calculations involving irrational numbers, as that would very easily display where the irrational number was rounded/truncated based on the number of digits the calculator uses. For example, two of my calculators (that I tested) give different answers for sqrt(2) - e + e - sqrt(2) (Done as three calculations. Root2-e then + e then -root2) (I get 0 for my Fx-991W and -3.7E-10 for my Fx-5000F Both give 0 when done in a single calculation) Admittedly, the algorithm used there is more "accurate" or rather shows differences to a higher degree of precision. (And theirs doesn't depend on the stored values for e.)

[corrado33]

Here's the reply from casio if you're interested.

Thank you for contacting Casio America. It is difficult for me to determine what might have caused this error, as the FX-5000F is a very old calculator (released in 1987), and we do not have a working unit on hand.

One possibility is that you may have entered the problem incorrectly into your calculator. I have tested the example that you provided on two recent calculator models, and both reported the roots as -0.15 and 5.133333333E-12 (not 5.1E-13). With that in mind, you may want to double-check the entry.

Another possibility is that there could have been an error in the precision. This may have been a limitation of the processing capability of the calculator from 28 years ago, compared to today's models. Unfortunately, there would be no way to address this at this point, since the FX-5000F has been out of production for many years.

We hope that we were able to assist you with your question, and again thank you for contacting Casio America. As a reminder we can also be reached at 1-800-435-7732 option 5.

[TimFox]

When dealing with the quadratic equation where you get the small difference between two larger numbers, if is often useful to make an approximation and get most of the large values to cancel directly, leaving you with what you want.
I wrote out these equations in Visio, making the usual approximation for the square root of unity plus a small number.