[Users] Data limitations and performance

[Paul de Groot]

Dear OpendTect users,

One of our users said:

>I have hence some questions for you and the use of the tool:
>...(I should use the discussion web for that) ...
>
Yes, please we encourage everbody to pose questions of general interest 
to the user community to the mailing list.

>- I am currently dealing with surveys of 2000 x 1000 traces, with 500
>samples. It is a lot, but it is rather common as survey size. Do you have
>any idea of the data limitations (if any) , and recommendations. Do you have
>any table showing the processing necessary time for heavy computations,
>say by machines types. 
>
Unfortunately, we don't have any benchmark test results. The simple 
speed tests we did confirmed what is generally known: Linux is faster 
than Solaris, dual processors are faster than single-processors and 
64-bit is faster than 32-bit processing. If anyone did a real benchmark 
(e.g. clocking the speed of calculating a steering cube with default 
settings on different platforms), please share this information with us.

Regarding the data size and potential limitations, I am pleased to 
inform you that as far as we can see - and as far as we have been able 
to test - there is no limit imposed by the software. There are of course 
limits imposed by the hardware - memory size, graphical card speed, hard 
disk speed. The very large surveys we have had here (for exampe 
4000x4000 traces x 1250 samples) show that it is not feasible to try to 
work on the entire survey at once for attribute evaluation. For this 
purpose OpendTect supports the "Work area" (View menu) concept.

In general, we can distinguish three main tasks for OpendTect:

1) Evaluation of attributes, picking training data, etc.
2) Processing to create entire volumes
3) Data visualisation

(1) Has problems specifically on larger time slices and horizons. For 
time-slices it is important how you stored the data (option "Optimize 
horizontal slice access") that is set when you import seismic data. The 
dis-advantage is that crossline retrievals will the be slower. (If speed 
is an issue and data storage is not you may consider to create two 
versions of the volume: one for time-slice retrievals and one for 
crosslines). To ease the pain for horizons use the store and retrieve 
attributes option.
(2) Has no real limits: add as many computers as you want  
(multi-platform, distributed computing is a key concept in OpendTect. If 
you add a cluster, all processors will be used).
(3) Many hardware-related limits and also some problems with 
non-optimised software.

We are aware of the fact that for large surveys, we have to do more work 
to really attack the big ones. Currently we are redesigning the 
attribute engine to take away some of the pain. In this domain there are 
two specific elements that could improve handling large data volumes: 
full bricking and compressed storage. One could think of a project 
'performance increase' that will try to isolate pain points and solve 
them but I'm afraid at the moment many other things compete for 
attention from the software department.

Best regards,

Paul.

-- 
-- Paul de Groot
-- OpendTect Support Team
-- paul.degroot at opendtect.org
-- http://www.opendtect.org
-- Tel: +31 534315155 , Fax: +31 534315104