- Getting Started
- Getting Your Data Into CASA
- Processing SMA Data in CASA
- Getting Help
Welcome to the SMA CASA Website.
The purpose of this site is to explain how SMA data can be processed with CASA. The information presented here covers how to get SMA data into the CASA package and SMA-specific CASA processing steps.
SMA data can be imported directly into CASA, without any pre-processing by any other package such as MIR or Miriad. It has been tested with single and dual receiver data, and data from the new SWARM (SMA Wideband Astronomical ROACH2 Machine) correlator. It cannot yet process polarization data.
CASA (Common Astronomy Software Applications) is fully documented at http://casa.nrao.edu.
Two python scripts have been written that allow you to import SMA data into CASA:
| This script produces a set of FITS-IDI files.
One such file is produced for each sideband of each spectral chunk in the
data set. This script is run at the shell level and writes the FITS-IDI files into the current working directory. If you run this script on one of the CF-managed linux machines in Cambridge, MA, the script should run without modification. To run the script elsewhere, you'll need to modify the scripts first line to point to your own local python distribution.
By default, the sma2casa.py script calls a C language module called "makevis". The object code for that module is makevis.so, and it must be present in the directory you are running sma2casa.py in. This module does the low-level processing of the visibilities data, and writing it in C rather than Python very significantly speeds up the execution of sma2casa.py. Since makevis.so is compiled code, it is not as portable as Python code. You may need to recompile makevis.c if makefile.so does not work properly on your machine. The source code and a Makefile are included in the git repository. It is almost certain that the Makefile will work only on Linux machines. If you have trouble with makevis, and don't want to go through the hassle of figuring out how to compile it, you can use the -P option for sma2casa.py. If -P is specified, sma2casa will not use makevis and will run using Python only, which will be significantly (factor of ~2) slower, but more portable.
|This script is run from the CASA ipython interpreter. It reads in the FITS-IDI files that sma2casa.py created, performs some calculations to make the SMA data more similar to ALMA data, and then concatonates everything into a single CASA Measurement Set (MS).|
The SMA CASA python scripts are updated frequently (mostly bug fixes). Before processing data from a new track users should verify that they have the latest versions of the scripts.
The most recent versions of these scripts can be obtained from the github software repository by issuing the command:
ShellPrompt> git clone https://github.com/kenyoung/sma2casa.git
git is free software for Linux and Mac machines. Free download is available at http://git-scm.com/
Python is free software for Linux and Mac machines. Free download is available at http://www.python.org/
SMA CASA scripts have been fully tested for Python 2.x; however, the SMA CASA scripts have not been tested with Python 3.x.
To determine the version of Python running on your machine, type the following:
ShellPrompt>python -V Python 2.7.3
There are several Python modules required in order to run the SMA CASA scripts:
NOTE: These modules listed under smaImportFix.py are part of a standard CASA installation.
To verify if these modules are installed, type the following commands:
If you are having trouble getting a self-consitent set of Python and Python modules together which will support sma2casa.py, you could consider installing the Anaconda Python distribution. That installation has everything needed, and sma2casa.py is tested using that distribution.
|Running CASA on the R&G or CF Computers or the Hydra Cluster|
There are six versions of CASA on the R&G public computers:
|CASA version||Executable in /usr/local               ||Variable name|
|3.0.0||casapy-30.0.9860-001-64b            ||casapy30  |
|3.1.0||casapy-31.0.13530-002-64b          ||casapy31  |
|3.3.0||casapy-33.0.16856-002-64b          ||casapy33  |
|3.4.0||casapy-34.0.19988-002-64b          ||casapy34  |
To run version 4.0.1 on RTDC7, type:
- ssh -X rtdc7
- ssh -X rtdc7
NOTE: Casapy is updated weekly via rsync.
Additionally, there are three versions of CASA on the CF managed linux computers:
|CASA version||Executable in /sma/ALMA               |
To run version 4.1.0 on cfa0, type:
- ssh -X 22.214.171.124 (if not on CF managed machine)
- ssh -Y cfa0
In the near future, users will be able to run their time or cpu intensive jobs using parallelized CASA on SAO's HYDRA cluster.
For a progress report, click here.
|Running CASA on the Hawaii Computer|
There is one versions of CASA in Hawaii on hilodr2.sma.hawaii.edu:
NOTE: To logon to hilodr2 you must obtain the IP address. Please contact smaCASA@cfa.harvard.edu for assistance.
|CASA version||Executable in /usr/local               ||Variable name|
To run, type:
- ssh ipaddress
- cd reduction
sma2casa.py creates one FITS-IDI file for each sideband of each chunk in an SMA data set.
Once you have obtained the python scripts needed to import SMA data (see "Getting Started" section), you should be able to convert your SMA data into a CASA Measurement Set (MS).
sma2casa.py does not directly create a CASA MS. Instead, it creates one FITS-IDI file for each sideband of each chunk in an SMA data set. The reason for this is that the CASA MS format can be changed by the CASA software group at any time, but the FITS-IDI format is a standard format which is unlikely to be changed (because that would break may software packages). So FITS-IDI provides a fixed target for sma2casa, and CASA itself provides a routine (which is called by smaImportFix) for reading FITS-IDI files.
sma2casa.py uses the Tsys information in the SMA dataset to convert the SMA visibilities to "pseudo-Jansky" units, which should be fairly close (within ~20%) to the correct Jy values, unless there were significant problems with the track (for example, bad pointing). It also uses Tsys, along with integration time, to calculate weights for the visibilities. These calculations are performed before the FITS-IDI files are written.
The only required pararameter for the sma2casa.py script is the path to the SMA-data set.
sma2casa.py accepts the following optional parameters:
|Usage: sma2casa.py path-to-SMA-data [options]|
|-c m,n,o [--chunks=m,n,o]||Process only chunks m, n and o|
|-h [--help]||Print this message and exit|
|-l [--lower]||Process lower sideband only|
|-p [--percent]||% to trim on band edge (default = 10)|
|-P [--PythonOnly]||Do not use the C module "makevis"|
|-r [--receiver]||Specify the receiver for multi-receiver tracks|
|-R [--RxFix]||Force the data to be treated as single receiver|
|-s [--silent]||Run silently unless an error occurs|
|-t [--trim]||Set the amplitude at chunk edges to 0.0|
|-T||-T n=m means use ant m's Tsys for ant n|
|-u [--upper]||Process upper sideband only|
|NOTE: The options list above must come after the path argument.|
The -T option provides a crude way to handle bad Tsys information. If antenna n has noisy or garbage Tsys values, -T allows antenna m's Tsys to be used instead for that antenna. Dual receiver tracks must use the -r option, and each receiver must be processed separately.
ShellPrompt> sma2casa.py /sma/SMAusers/taco/130408_17:20:01/
will process both sidebands of all chunks in the data set located at /sma/SMAusers/taco/130408_17:20:01/
NOTE: The command above will fail if that data set has data from more than one receiver. To reduce dual receiver tracks the -r option must be used to select a receiver..
ShellPrompt> sma2casa.py /sma/SMAusers/taco/130408_17:20:01/ -u -r 400
will process the data from the upper sideband of the 400 GHz Rx only.
ShellPrompt> sma2casa.py /sma/SMAusers/taco/130408_17:20:01/ -l -c 3,5,20
will make a FITS-IDI file for the lower sideband of chunks s03, s05 and s20, if data exists for those chunks (i.e. if the number of channels has not been set to 0 in the restartCorrelator command).
ShellPrompt> sma2casa.py /sma/SMAusers/taco/130408_17:20:01/ -l -c 40 -t
will make a FITS-IDI file for the lower sideband of chunk s40, and trim the highest and lowest 10% of the channels by setting their amplitudes to 0.0 (which will ultimately cause them to be flagged bad).
ShellPrompt> sma2casa.py /sma/SMAusers/taco/130408_17:20:01/ -T 4=7 -T 5=7
will make a set of FITS-IDI files with the Tsys values for antennas 4 and 5 replaced by the Tsys values for antenna 7.
The sma2casa.py script calls a C language module called makevis. The object code for that module is makevis.so, and it must be present in the directory you are running sma2casa.py in. This module does the low-level processing of the visibilities data, and writing it in C rather than Python very significantly speeds up the execution of sma2casa.py. Since makevis.so is compiled code, it is not as portable as Python code. You may need to recompile makevis.c if makefile.so does not work properly on your machine. The source code and a Makefile are included in the git repository. It is almost certain that the Makefile will work only on Linux machines.
If are having trouble with makevis, and don't want to bother getting it working on your machine, you can specify -P on your sma2casa.py command line, which will make the script run using Python code only, and be a factor of ~2 slower.
|Host Computer||sma2casa.py processes the visibilities by mapping the entire visibilities data file into RAM. The script is apt to run very slowly if the computer's available RAM is smaller than the size of the file sch_read.|
|Missing tsys_read File||If you run sma2casa.py on a very old SMA data set, it may immediately abort because the data set does not contain a tsys_read file. This file is required, and it can be built from data stored in the eng_read file. This process can't be done automatically, because the procedure depends on the receivers which were active in the track, and the state of the Bandwidth Doubler Assembly. Contact smaCASA@cfa.harvard.edu if you need to have a tsys_read file made for your track.|
|Flux Scale||The pseudo-JY scale produced by sma2casa.py has amplitudes ~15% higher than what apply_tsys produces in MIR. Of course, once you perform flux calibration, the pseudo-Jy scale will be wiped out anyway, so point is of interest only if you don't flux calibrate.|
smaImportFix.py reads the FITS-IDI files into CASA.
The second Python script, smaImportFix.py, will read the FITS-IDI files into CASA, and produce a single MS with all available chunks, including the pseudo-continuum "chunk". smaImportFix.py should be executed from within CASA, and it assumes that the FITS-IDI files have already been created by sma2casa.py, and that those files are in the current/working directory (pwd). The script does the following things:
- Makes a list of which FITS-IDI files are in the current directory, so that it knows which chunks should be processed.
- Reads each FITS-IDI file into a separate CASA MS.
- sma2casa.py indicates a data value is bad by setting its amplitude to 0.0 . smaImportFix.py runs the flagdata task on the newly created MS, in order to explicitly flag those data points bad within the MS. Chunk edge channels are also flagged bad in this step, if you passed arguments to sma2casa.py indicating that you wanted to trim edge channels.
- The FITS-IDI files are deleted.
- Fixes the weights. For all chunks except the pseudo-continuum chunk, the CASA importfitsidi file sets the data weights to 1.0. smaImportFix.py fixes this problem, and puts the proper weights, proportional to (integration time)/Tsys**2, in the weight table of the MS.
- Generates new scan numbers. In the raw SMA data sets, each timeslice of data stored by the correlator has a unique scan number. CASA MSs usually have scan numbers which change only when the source is changed, which can be helpful in controlling how calibration information is averaged and interpolated So smaImportFix.py generates a new set of scan numbers which increment only when the source changes. This means that there will usually be several integrations which share the same scan number.
- The individual chunk MSs are concatonated into a single MS. There is one such concatonated MS made for each sideband, named MyDataLower and MyDataUpper.
There is a strange, intermitent problem with importing Dual-Rx data into CASA. Occasionally the frequency scale gets set incorrectly. There is a work-around for this issue:
|System Temperature Table||The CASA SYSCAL table has the Tsys values stored in it, but that is probably only useful for plotting the Tsys data (via browsetable). The SYSCAL table is not in the format expected by CASA, so the CASA commands which use Tsys information will not work properly.|
|Chunk Names in CASA||If you import all the spectral chunks, but not the pseudo-continuum "chunk" (this is the default behavior) then chunk s01 will be spw0, chunk s02 will be spw1 etc.|
|Antenna Names and pads||The names of the antennas in CASA will be "SMA1", "SMA2", etc. The "Station" parameter for each antenna is set to the pad name for the pad the antenna was sitting on during the observation.|
For help processing SMA data in CASA please contact smaCASA@cfa.harvard.edu .
This website is intended to provide information to users in order to process SMA data using CASA.
General information regarding CASA is available at the National Radio Astronomy Observatory (NRAO) website http://casa.nrao.edu.
The SMA CASA staff/scientist/users welcome your comments and encourage you to join our mailing list smaCASA@cfa.harvard.edu
|Notification of SMA CASA script updates|
|Submitting error reports|
|Tips, Shorts-Cuts, etc.|
|Data testing and anaylsis|