[OpendTect_Developers] When floats are not enough

[Bruno Chaintreuil]

Hi All,

I just fixed a bug in OpendTect which was caused by accumulating 
uncertainties when using 'float' values. I thought it might be 
interesting to share this with you. In general, the precision given by 
the 4 bytes floating point is enough for most position calculus. But in 
some cases it might lead to approximate or simply wrong results.

I had a bug report from our case study department: Arnaud tried to 
compute a variance on a surface for a histogram display and ended up 
with... a negative result! Looking at the code, nothing seemed to be 
wrong. Our Stats::RunCalc algorithm (/include/Algo/statruncalc.h) was 
used with the type float to compute the variance with the formula :

Variance = ( ( Sum(Xi²) - Sum(Xi)² / n ) )/ (n-1)

After a bit of debugging I realized that while rising the float values 
to the power of 2 to compute the sum of the squares in the first term, 
some precision was lost because the x values had lots of digits after 
the decimal point. Therefore for a high rank of the index i, the sum of 
the squares became inferior to the square of the sum divided by the 
total number n, hence the negative value. I solved the problem by 
initializing the template of the Stats::RunCalc with the type 'double' 
(8 bytes floating point) instead of 'float'.

Of course, this does not mean that you always have to use double 
floating points everywhere, but just keep in mind that when a high 
precision is needed, you might want to use doubles.

Best Regards,
Bruno

-- 
 Bruno Chaintreuil, MSc.
 Software Engineer
 dGB Earth Sciences B.V.
 Nijverheidstraat 11-2, 7511 JM Enschede, The Netherlands
 bruno.chaintreuil at dgbes.com
 Tel: +31 534315155 , Fax: +31 534315104
 http://www.dgbes.com