[OpendTect_Developers] Remote 3D rendering/ Use of the GPU

Wed Mar 6 18:32:33 CET 2013

Dear all,

So I will give a go on a small HowTo virtualise OpenGL rendering/GPU calculations for Opendtect.

I used the combination of VirtualGL server 2.3.2 and TurboVNC 1.1.90 on a dual xeon server with lots of RAM and a NVidia Quadro. The OS is Ubuntu 12.04 (Pangolin). For Quadro cards avoid the new Ubuntu version (12.10), it is not yet approved.

I principally used the documentation from VirtualGL which is fairly complete:
http://virtualgl.svn.sourceforge.net/viewvc/virtualgl/vgl/tags/2.3.2/doc/index.html
And TurboVNC:
http://virtualgl.svn.sourceforge.net/viewvc/virtualgl/vnc/tags/1.2beta1/doc/index.html

============

First you can install TurboVNC from the packages available at sourceforge. I have a debian-based system so I did (after download):
sudo dpkg -i turbovnc*.deb

Then I ssh my server (I did it from my own server):

/opt/TurboVNC/bin/vncserver

The first time you have to set up a password (Which I did). It will be asked each time you want to connect to the X11 proxy (the vnc server).
You should also read this output.

Desktop 'TurboVNC: my_server:1 (my_user)' started on display my_server:1

This is very important because it tells you where to connect your client (s). The 1 refers to the virtual display number. To check if it is up normally you can kill this proxy with:

/opt/TurboVNC/bin/vncserver -kill :1

This command is very useful because it is possible that you loose contact with your server. The idea is that your client problems do not affect the server so the proxy runs whatever happened to your client.

If you want to be sure, before starting using a vnc viewer, ssh your server and type the command:

/opt/TurboVNC/bin/vncserver -list

N.B.: Personally, I want to log onto my server of which I am the only user. I work remotely through a VPN when I am travelling. So I can kill any remnant proxy. I suppose that people which want to share a Tesla rack or something similar to share between clients should use more advanced vncserver management options.

============

Secondly, you have to install the VirtualGL server
  * Once again I downloaded the binaries on sourceforge, then installed the package:

dpkg -i virtualgl*.deb

  * Now, you have to log out of your sessions (preferably kick out any potential user).
Alt+F1 to get a non X session.

sudo stop lightdm

There you can have various weird behavior but reopen another session if necessary, me it worked.

  * Then what is not in the help but in the script; use the command:

sudo rmmod nvidia

That really stops the X server. This is above my comprehension of the Ubuntu system but it worked. Without that the first time I set up the server, I lost access locally to my X session (.Xauth corrupted).

  * Then you should configure the Virtual GL server

sudo /opt/VirtualGL/bin/vglserver_config

Personally I replied yes to the 3 security restrictions

  * Add yourself and/or root to the line vglusers in the file /etc/group
NB: I did without at first and it worked too.

  * Supposedly at this stage you can re-start your display manager and everything runs smoothly with:

start lightdm

However, unity does not really like it and I had to really reboot the machine to get things back normally.

In parallel, there are sanity checks proposed in the documentation that you can find at the end of this paragraph:
http://virtualgl.svn.sourceforge.net/viewvc/virtualgl/vgl/tags/2.3.2/doc/index.html#hd006001

At that stage the server should be ready and there is no need of further configuration except if you want to set up several graphic card and other complexities.

==========
Set up a client:

  * You need to install the vnc viewer on the client machine via sourceforge. I tested the Debian and the Windows versions and both worked for me. No need of extra fuss, simplest is the best.

* ssh the server from your client and start a vnc server with the command:

/opt/TurboVNC/bin/vncserver

Write down or copy the server:display parameters from the output you got. Should be like:
Desktop 'TurboVNC: my_server:1 (my_user)' started on display my_server:1

You can then close the ssh connection if not needed.

N.B.: If you lost the connection check up if there is not a server you previously opened and kill it if necessary (Cf. first section).

  * On the client machine start the vncviewer. you will be prompted for a server. Paste or write down the server:display value. In our example it is:

my_server:1

Depending on the configuration settings, you will be prompted for the vncserver password that you set up in the first phase of the installation.
Then magic you will see appearing the server desktop which should be amasingly fluid in rendering compared to a regular ssh + local X server.

  * Then use VirtualGL to start opendtect by typing:
/opt/VirtualGL/bin/vglrun tcsh /OpendTect/4.4.0/start_dtect
With in my case, tcsh /OpendTect/4.4.0/start_dtect being the normal way for me to start the OpendTect system.

I have tested that in several conditions from a personnal wifi connection in the countryside in England toward a VPN redirecting to my personal machine at the Uni of Copenhagen and it runs perfectly smoothly. It is magic!

I hope this will help a bit to set up a server and you won't waste a couple of day like me to understand what is going wrong. It was mainly my mistake to read more the help than the script outputs but...

Best regards,

Julien

Julien Moreau, PhD
Assistant Professor, Applied Geophysics
Department of Geography and Geology, University of Copenhagen,
Øster Voldgade 10, 1350 København, Denmark
Office: +45 353-24168

________________________________
From: Kristofer Tingdahl [kristofer.tingdahl at dgbes.com]
Sent: 02 March 2013 23:23
To: Julien Moreau
Cc: developers at opendtect.org
Subject: Re: [OpendTect_Developers] Remote 3D rendering/ Use of the GPU

Dear Mr. Moreau,

thanks for sharing your experiences with VirtualGL. I tried to used 5 years ago (or so), but failed, so I'm glad you got it going.

Best regards,

Kristofer

On 28 February 2013 19:31, Julien Moreau <julien.moreau at geo.ku.dk<mailto:julien.moreau at geo.ku.dk>> wrote:
Thank you all for the advices,

I managed, after some crisis with my server, to make VirtualGL /TurboVNC perform properly. If the configuration fails, you can loose access to X locally (.Xauthority gets corrupted). The documentation is not completely up to date regarding lightdm and VirtualGL. However the scripts 'speak' the truth so that if you follow their verbose it actually works.

If it will be directly included into OpendTect it is indeed easier for the user and the sys admin. If I have a little bit of time, next week I will produce a small text for the ones which would be interested in doing the same.

Thanks for the help,

Julien
________________________________
From: Kristofer Tingdahl [kristofer.tingdahl at dgbes.com<mailto:kristofer.tingdahl at dgbes.com>]
Sent: 26 February 2013 21:12
To: Jb West
Cc: Julien Moreau; developers at opendtect.org<mailto:developers at opendtect.org>
Subject: Re: [OpendTect_Developers] Remote 3D rendering/ Use of the GPU

All:

One of the long-term projects we are working on at dGB Earth Sciences in converting our visualization from using Coin3D to OpenSceneGraph. It has turned out to take longer time than anticipated, but we are aiming at solving the remote visualization problem. Our solution is to have a special mode (invisible to the user) where we will render on the remote computer into a memory buffer, and then send the image over to the client side. It is what VirtualGL and TurboVNC is doing, but we are building the functionality into OpendTect. This will require high bandwidth between the client and the server, but the images should have the full quality.

Now, this is at least a year ahead before we are going to release anything with OSG, but I do want to give you a heads up.

Best regards,

Kristofer

On 26 February 2013 11:57, Jb West <jbwest at luminterra.comcastbiz.net<mailto:jbwest at luminterra.comcastbiz.net>> wrote:
Julien,
  Remote OpenGL over-the-wire is doomed to not work. Have a look at VirtualGL and the TurboVNC project. I've found this to be the best free remote-opengl solution. http://www.virtualgl.org/

JB West
jbwest at luminterra.com<mailto:jbwest at luminterra.com>
________________________________________
From: developers-bounces at opendtect.org<mailto:developers-bounces at opendtect.org> [developers-bounces at opendtect.org<mailto:developers-bounces at opendtect.org>] On Behalf Of Julien Moreau [julien.moreau at geo.ku.dk<mailto:julien.moreau at geo.ku.dk>]
Sent: Tuesday, February 26, 2013 2:57 AM
To: developers at opendtect.org<mailto:developers at opendtect.org>
Subject: [OpendTect_Developers] Remote 3D rendering/ Use of the GPU

Hello OdT Community,

I am a new to this list and I am not sure it is the right place to ask the question. However, I assume that you are probably the ones which are most likely to try this kind of operation.

For obvious economic reason and practicality I want to run OpendTect remotely on my workstation or in the future maybe a Cuda system or a server with Tesla cards. I work with OdT everyday but I want also to teach in class and convincing my Uni  to buy 20 quadro cards is not easy (So far I have 10 which is good).

I have performed several tests and from an old Dell laptop with a Quadro and Ubuntu installed I can connect on my workstation and it works perfectly fine through SSH.

Then I tried from a windows 8 laptop with a regular mobile NVidia + the intel chipset. Using Putty and Xming. I can do everything I want except indirect rendering. I spent some time reading about the error code (Xlib is missing the extension NV-GLX). And apparently it is due to the difference in architecture between the systems. There are compatibility flags and if the X server is not happy then you switch from indirect rendering to direct rendering (which is catastrophic over the network indeed). Then I went to see how to update my server in case it would perform better. And I discovered the numerous flavours of mesa and I must admit I don't know what to choose (I have by default installed mesa-dri).
Here a link to the description of the problem
http://www.linuxquestions.org/questions/linux-software-2/remote-3d-is-it-possible-839699/
I have also noticed the presence of a od_glxinfo program which might be some of the solution.

My investigation might seems confused and I would be please to have your advices on that. Particularly if you found a turnaround (knowing that I am not a huge fan of remote control of the machine through vnc).

Thank you in advance,

Julien

_____________________________________________________________
OpendTect Developers mailing list Developers at opendtect.org<mailto:Developers at opendtect.org>
http://lists.opendtect.org/mailman/listinfo/developers

You receive this mail because you are listed on developers at opendtect.org<mailto:developers at opendtect.org> To unsubscribe please go to http://lists.opendtect.org/mailman/options/developers . If you encounter any problems, please contact support at opendtect.org<mailto:support at opendtect.org> .

--
Kristofer Tingdahl, Ph. D.
CEO
dGB Earth Sciences

--
Kristofer Tingdahl, Ph. D.
CEO
dGB Earth Sciences

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opendtect.org/pipermail/developers/attachments/20130306/1ae53b14/attachment.html>