SMA Data Reduction
SMA data is collected in native MIR/IDL format and has until recently relied exclusively on MIR software for calibration. With the popularity of CASA in radio astronomy, new tools are becoming available allowing users to calibrate and/or image their SMA data using CASA.
The SMA does not currently offer users calibrated or imaged data products. We do however support a number of pathways for users to process their data independently. Note that extra steps are required for polarization data - scroll down for details.
Once you have retrieved your raw data, you should choose one of the following reduction pathways.
| Pathway | Rechunk (optional) | Convert to CASA | Calibrate with | Convert to CASA | Image with |
|---|---|---|---|---|---|
| 1 | pyuvdata | pyuvdata | CASA | - | CASA |
| 2 | SMARechunker | - | MIR | MIR autofits | CASA |
| 3 | SMARechunker | MIR mir2ms | CASA | - | CASA |
| 4 | SMARechunker | - | MIR | - | MIRIAD |
Find data reduction tutorials here.
Rechunking SMA data
Rechunking, or rebinning, can be performed by SMARechunker or pyuvdata. Pyuvdata can additionally be used to transform the format of raw SMA, most commonly to CASA MS. Since the introduction of the SWAMR correlator in 2016, SMA data has a fixed resolution of 16 kHz. The bandwidth, while also fixed, as been growing as SWARM has been expanded to wide-band mode. This means file sizes in excess of 100 GB are now fairly common. Unless you need high spectral resolution for your science goals, you will find the smaller sizes of rechunked data far quicker to process.
More information:
Reducing the Size of your Data
Converting between MIR and CASA
There are two routines available to convert SMA data to CASA measurment set.
1) For uncalibrated/raw data use pyuvdata or the MIR command mir2ms.
2) For data calibrated using MIR, use the MIR command autofits.
More information:
Converting SMA Data to a CASA Compatible Format
pyuvdata with the SMA
Calibrating SMA data
SMA data can be calibrated in either MIR or CASA and both are routinely used. MIR has the advantage of having been designed specifically to handle SMA data and runs extremely fast, however, it does require an IDL licesnse and requires significant amounts of memory ~ 2-3x the data size.
CASA may be more familiar to some users, but users should expect calibration routines (especially bandpass) to take considerable timecto complete depending on the file size.
If you choose to calibrate your data in CASA you will need to convert the raw SMA data. Your options are to use pyuvdata or the MIR routine mir2ms. Remember that MIR requires an IDL license. Pyuvdata is a powerful tool to manipulate UV data, it can change formats and rechunk/rebin data in the process.
More information:
SMA Data Calibration with MIR
The MIR Cookbook
Using CASA on the RTDC
Accessing CASA on the RTDC
Imaging SMA data
MIR calibrates SMA data but does not perform imaging tasks (inverting, cleaning etc). Until recently, imaging was most commonly performed by MIRIAD and some users still choose this option for the exceptional speed. However, MIRIAD is now unsupported for SMA data and we encourage you to use CASA.
More information:
Using CASA on the RTDC
The special case of polarization data
There is currently no pathway to reduce SMA polarization data in CASA. Polarization data must be calibrated in MIR, followed by an additional step in MIRIAD. From there the data can be written out in fits format and imaged with your prefered program (MIRIAD or CASA).
If users are interested in the source and not concerned with the cross polarization terms, then polarization data can be treated as regular total-intensity data.
Users wishing to reduce polarization data should email Holly Thomas for assistance.
More information:
SWARM Polarization Calibration Tutorial
SMA Data Calibration with MIR
What about older data?
The correlator preceding SWARM was known as ASIC. After a couple of years of overlap, ASIC was taken offline completely in 2016 (see details at SMA Data Format). We do not recommend using CASA for imaging older ASIC data.
More information:
ASIC (2003-2016) Calibration Tutorial
SMA Data Reduction with MIRIAD
MIRIAD Homepage
