R: [Developers] Parallelization of computations

[Corrao Antonio]

Hi,
regarding parallelism, I guess most of my plugins can not exploit 'independent samples' multithreading the way the attached API would allow; nonetheless, any two distinct <il,xl> locations are independent, so I could imagine computeData being called for distinct traces in separate threads.

Said that, I noticed that process_attrib does not spawn multiple threads (they should show up as distinct processes in ps -efwww) and top reports that the two CPUs are not contributing equally: one is idle and one 99% busy.

Now, is it a platform issue (OS, kernel version, etc..) or just that multithreading is not implemented (yet) the way I would suggest? 

Best regards,
AC


Antonio Corrao
Senior Geophysicist
GEOS dept. - Eni E&P Division
V. Emilia 1 - 20097 San Donato Milanese - Italy

> -----Messaggio originale-----
> Da: developers-bounces at opendtect.org [mailto:developers-
> bounces at opendtect.org] Per conto di Kristofer Tingdahl
> Inviato: giovedì 3 gennaio 2008 17.30
> A: developers at opendtect.org
> Oggetto: [Developers] Parallelization of computations
>
> Dear fellow developers,
>
> over the last two years we have seen a new development in the
> cpu-development. Earlier, the focus was to increase calculation speed by
> increasing cpu-frequency, and multiple cpus were expensive and only
> available on high end machines. Today, however, most workstations and
> computation nodes has at least two cpu-cores, and within a year four our
> eight cores will be very common. This development urges software
> development to be able to utilize this computation power by computing on
> multiple cpus in parallel.
>
> OpendTect has great support for parallelization of computations, and I
> will highlight parallelization of attributes in this e-mail.
>
> Attribute calculations are very easy to adapt to parallel processing.
> All attributes inherits the class Attrib::Provider, which has the
> virtual functions:
>
> bool allowParallelComputation() const
> bool setNrThreads()
> int minTaskSize()
>
> If your algorithm is sample independent for each trace, that is that the
> order in which the samples on a trace are computed does not matter, you
> can turn on parallel computation by implementing
> allowParallelComputation() to return true. Examples of attributes that
> are sample independent are similarities, energies and so forth. Examples
> of attributes that are not sample independent are unwrapped phase, where
> the angle increases from the top of the trace.
> The function setNrThreads is called on the object to notify it how many
> trace will be used. This is useful if you need to setup datastructures
> for each thread, e. g. you have an fft that cannot be shared between
> different threads. in computeData(), you will get a threadidx so you
> know which fft to use.
> The function minTaskSize() is the smallest number of samples that will
> be computed in a single thread. If your trace is short, and the
> algorithm is fast, it may not be worth the effort to run it in a large
> number of parallel computations. Then you return a large number in
> minTaskSize. On the contrary, if the algorithm is slow, it may be worth
> the effort to divide the computation in many parts, even if each part
> only has a few samples to compute.
>
> If you have any questions about this, or other OpendTect development
> issues, don't hesitate to contact dGB or (even better) send it to this
> list.
>
>
> Best regards,
>
>
> Kristofer Tingdahl
>
> --
> Dr. Kristofer Tingdahl
> dGB Earth Sciences
> 1 Sugar Creek Center BLVD #935; Sugar Land, TX 77478; USA
> +1 281 652 5318
>
>
> _______________________________________________
> Developers mailing list
> Developers at opendtect.org
> http://lists.opendtect.org/mailman/listinfo/developers

------------------------------------------------------------------------------------------------------------------------------------------
Message for the recipient only, if received in error, please notify the sender and read http://www.eni.it/disclaimer/