LotusScript isn't the only language that struggles with floating point
Category javascript
Jerry Carter recently mentioned that if you're doing raw calculations using floating point (i.e. using the data types Double or Single - which are just short for, respectively, double-precision floating point and single-precision floating point) numbers, you may not get the results you're expecting. A lively conversation ensued.
This morning, Ajaxian covered an article from Douglas Crockford that explains why JavaScript suffers from the same problem (they also hinted that Java does as well). Moral of the story: if your calculations must be precise, don't rely on floating point, no matter what language you're using (unless the language includes a specific data type that does the necessary logic for you automatically, such as the Currency type in LotusScript). Rather, nudge your scale up to the point that all operands involved are integers (i.e. multiply all of them by 100, 1000, etc.), perform the operation, then divide by the same scale to get the correct result.
Jerry Carter recently mentioned that if you're doing raw calculations using floating point (i.e. using the data types Double or Single - which are just short for, respectively, double-precision floating point and single-precision floating point) numbers, you may not get the results you're expecting. A lively conversation ensued.
This morning, Ajaxian covered an article from Douglas Crockford that explains why JavaScript suffers from the same problem (they also hinted that Java does as well). Moral of the story: if your calculations must be precise, don't rely on floating point, no matter what language you're using (unless the language includes a specific data type that does the necessary logic for you automatically, such as the Currency type in LotusScript). Rather, nudge your scale up to the point that all operands involved are integers (i.e. multiply all of them by 100, 1000, etc.), perform the operation, then divide by the same scale to get the correct result.

Comments
Posted by Kerr At 01:19:07 PM On 03/16/2009 | - Website - |
Posted by Richard Schwartz At 02:40:40 PM On 03/16/2009 | - Website - |
Posted by Colin Macdonald At 03:36:29 PM On 04/08/2009 | - Website - |