How to use IDL & GDL on the cluster
- IDL is available on
hydra, and uses the pool of the licenses available on the CF-managed machines.
- Since IDL checks out a license when started:
- It is not meant to be used on a cluster (i.e., running a slew of IDL jobs), as you will drain off the licenses, run out of licenses, and, prevent all users (on
hydra and at CfA) from running IDL.
- We have a set of special licenses, (i.e., 128 runtime licenses), to enable using IDL in batch mode, meant for use on the cluster.
- The Grid Engine scheduler will keep track of license use to limit the number of concurrent use, but only if you add the
-l idlrt=1 flag to you submission (see below).
- IDL runtime has a few minor restrictions, check IDL's manual
- IDL (interactive/regular and runtime) will fail to get a license if
- the connection to CfA is down
- the license server at CfA is down
- GDL is also available on
hydra, ver. 0.9, see below.
GDL is a open source package that is compatible with IDL (v 7.1), see http://sourceforge.net/projects/gnudatalanguage/
Where is IDL on the cluster
- IDL is installed on
/software/rsi/ (instead of
/opt/rsi on CF-managed machines).
- Matt Phelps has installed versions 6.1 and 6.4 on
hydra (a while back).
- Examples to configure your environment for IDL are in
csh, respectively), look at
~hpc/.cshrc for examples on how to use them. Make you own copy, adjust it if needed, and edit accordingly your
hydra uses the CF licenses, we could install versions 7.0 and 8.1.
Email me (at hpc@cfa) if you need a more recent version, and let me know why.
How to use IDL runtime licenses
- The run-time licenses are for IDL ver. 6.1 (the latest version at the time we purchased them).
We decided not to pay for s/w maintenance on these.
- You need to compile all of your IDL code, including all the routines that may be called, into a single file called a
You can produce that
save set on any machine, but you must use the same version, i.e., 6.1.
- Your top level cannot be a script (aka
MAIN), but must be a routine (i.e., a
PRO), that needs no arguments (since they cannot be set).
- To produce the
save set, let's assume your top level routine is
reduce and is in
reduce.pro, execute the following:
IDL> .run reduce
IDL> save, /routine, file='reduce.sav'
- After this, you only need the
.sav file on
hydra, and any changes you would make to the
.pro will not affect the execution of the
.sav file, until you regenerate it.
- To run
IDL and pull a runtime license, your start
IDL with the
-rt flag, hence your job file should look something like:
#$ -l idlrt=1
- You can test you
.sav file, by running it interactively, using the
-rt switch, on any CF-managed machine, or for a short test run on
hydra 's head node.
- The name of the
.sav file must the same as the top routine to run.
- If the
.sav is in a different subdirectory, IDL will
cd to that subdirectory when started.
- Tell IDL to use only one CPU, since all nodes have multiple cores and IDL will, by default, multi-thread large computation, and use that way more than 100% of one CPU. So add the following line in your top procedure:
CPU, TPOOL_NTHREADS = 1
- We have only 128 runtime licenses, therefore, when submitting a job that will use an IDL runtime license, you have to add the flag
-l idlrt=1 to your
qsub command (or add the embedded directive
#$ -l idlrt=1 in your job file(s).) This will let the job scheduler keep track of license use and limit the number of concurrent running jobs. Still consider limiting the number of concurrent jobs you submit, since it is a shared resources (and thus avoid draining them by a single user).
- IDL license sever is dumb and will pull regular (interactive) licenses if:
- you run out of runtime licenses,
- the regular IDL license server is running but the runtime server is for some reason down (it has happened).
- You cannot prevent using 'regular' license, but by telling the scheduler that you pull a license it will limit the number of concurrent jobs.
The following line in your top procedure (see examples below):
IF lmgr(/runtime) THEN message, /info, 'runtime mode is on' $
ELSE message, 'not in runtime mode'
only tells you how you started IDL, not
what type of license was pulled.
You can find two examples, on
. Look at
These are converted to
So copy the
files and produce the save sets with:
hydra% idl c-test_me
hydra% idl c-sleep_two
then test them on the head node with:
hydra% idl -rt=test_me.sav
hydra% idl -rt=sleep_two.sav
allows you to check (w/
, as described above) that you have indeed pulled a runtime license, while waiting for you to press
will print the date, pause for two minutes, print the date again and finish.
It checks whether it is pulling a runtime license or not, i.e.
hydra% idl -rt=sleep_two
The job and qsub files, to run
respectively, note the embedded flags in the job file.
To submit a job to run
- Create the
.sav file with
hydra% idl c-sleep
- Submit it with
hydra% source sleep.qsub
- It will run, once a license is available, and will produce the log file
Monitoring license use, on HYDRA:
- to check the current limit on concurrent IDL runtime jobs:
hydra% grep idlrt_license /opt/gridengine/default/spool/qmaster/exec_hosts/global
- to check how many IDL runtime licenses are being used:
hydra% qhost -F idlrt | head -4
- To find out which users are using or requesting R/T licenses:
this command can produce a long output, so redirect the output to a file and search for the string
hydra% qstat -r -u '*'
- I have updated
q+, my PERL wrapper around
alias q+ ~hpc/sbin/q+), and added the
-lic option to help track license use.
hydra% q+ -lic
Monitoring license use, at SAO, on a CF-managed machine
- You can check who many runtime licenses are available with:
cfa0% /opt/rsi/idl_6.4/bin/lmstat -c /opt/idl/license/license.dat -f idl_rt
- Each instance of
idl -rt=file.sav that checks out a license will grab 6 tokens (for
linux), we have 768 tokens (6 * 128).
So if 240 are "in use", there remains 528 tokens for 88 instance of
Note that for IDL (FlexLM), an instance is when the combination of the user name, host name, and display name are unique.
How to use GDL
- To use GDL on
hydra, setup your
LD_LIBRARY_PATH to include
/share/apps/plplot/lib, the executable is in
- An example of setup could be (
csh syntax, assumes
LD_LIBRARY_PATH is already set):
setenv LD_LIBRARY_PATH $LD_LIBRARY_PATH":/share/apps/plplot/lib"
alias gdl /share/apps/gdl/bin/gdl
- GDL does not use any license, since it is open source, but you get what you paid for.
- GDL is installed on CF-managed machines under
/usr/bin/gdl, and on these machines you don't need to set
LD_LIBRARY_PATH to a specific value.
- 30 Jan 2012/Revised 26 Feb 2013