News Score: Score the News, Sort the News, Rewrite the Headlines

Intel Underestimates Error Bounds by 1.3 quintillion

Intel’s manuals for their x86/x64 processor clearly state that the fsin instruction (calculating the trigonometric sine) has a maximum error, in round-to-nearest mode, of one unit in the last place. This is not true. It’s not even close. The worst-case error for the fsin instruction for small inputs is actually about 1.37 quintillion units in the last place, leaving fewer than four bits correct. For huge inputs it can be much worse, but I’m going to ignore that. I was shocked when I discovered t...

Read more at randomascii.wordpress.com

© News Score  score the news, sort the news, rewrite the headlines