The RTDC
Processing SMA Data
1.2 m Telescopes
AST/RO
Extra

Converting SMA data to CASA MS format


  1. Using MIR   (RECOMMENDED)
  2. Using sma2casa.py   (UNSUPPORTED)


1.   Using MIR

This option converts data already calibrated in MIR to CASA MS format. The data are saved as UVFITS in MIR, then the provided script used to import it correctly into CASA.

  • Retrieve script
    You can retrieve the script from MIRFITStoCASA.py This script is used in place of CASA's importuvfits procedure which does not propagate the weights correctly from MIR.

  • Status
    [July 2019] A bug in fits_out (found inside the MIR autofits routine) has been fixed. This caused each chunk and sideband to have slightly different uv coordinates (meters) in casa. This problem meant users saw a narrower u-v coverage per baseline, and got u-v coordinates wrong by a few percent.

    [Mar 2018] MIR's fits_out routine (called by autofits) now outputs correct sky frequency header information for continuum uvfits files. Spectral data are unaffected.

  • Instructions
    1. Create the uvfits file in MIR by using the autofits routine. This will loop over all chunks, sidebands and receivers on a source-by-source basis. In this example our source is named orion.

      IDL> select,/p,/re
      IDL> autofits, source='orion'

      after providing your source name, autofits creates a separate file for each SWARM chunk (s1-s4), sideband, and receiver. The files are named with the following convention SOURCE_SB_CHUNK_RX.UVFITS   (e.g. ORION_L_S1_RX345.UVFITS, ORION_U_S4_RX230.UVFITS).

      For a typical SWARM data set you will get 16 spectral files, along with 4 extra files for the pseudo-continuum chunks, C1, which can be ignored.

    2. Next switch to CASA and use the provided script MIRFITStoCASA.py to import your data. This script loops over all the .UVFITS files and converts each of them to the measurement set (.ms) format.

      CASA: import sys 
      CASA: sys.path.append('/path/to/MIRFITStoCASA.py/') 
      CASA: import MIRFITStoCASA 
      CASA: fullvis='allorion.ms'
      CASA: allNames = []
      CASA: for sou in ['ORION']:
      CASA:    for rx in ['345','240']:
      CASA:       for sb in ['L','U']:
      CASA:          for i in ['1','2','3','4']:
      CASA:             name = sou+"_"+sb+"_S"+i+"_RX"+rx
      CASA:             print("------converting "+name+" ....")
      CASA:             MIRFITStoCASA.MIRFITStoCASA(UVFITSname=name+'.UVFITS', MSname=name+'.ms')  
      CASA:             allNames.append(name+'.ms')
      

    3. Next concatenate all the newly created measurement sets into a single file. The input here is allNames which is the python list of .ms files created in step 2. The name of the concatenated output file (fullvis) was also defined in step 2.

      CASA: concat(vis=allNames,concatvis=fullvis,timesort=True)

    4. Check the content of the final, concatenated measurement set:

      CASA: listobs(fullvis)

    5. Flag the noisy edge channels.

      CASA: flagdata(vis=fullvis, mode='manualflag', spw='*:0~nflagedge;(ntotal-nflagedge)~(ntotal-1)')

      where nflagedge should be substituted with the integer number of channels you want to flag from the edges, and ntotal should be substituted with the integer number of channels per chunk (or spw in CASA). ntotal can be found from the previous listobs command.

      The choice of how many edge channels to flag can be made by looking at the amplitude behavior as a function of frequency in each chunk, e.g. by using CASA's plotms function. Note that this can be slow if a lot of channels are present and/or a lot of tracks have previously been combined:

      CASA: plotms(vis=fullvis, xaxis='channel', yaxis='amp', avgtime='1e20', avgscan=True, iteraxis='spw')

      This will show increased noise in the edge channels. We advise a conservative trim of about 8% (so that nflagedge~0.08ntotal).

    6. Repeat the loop to create a new measurement set for another source. Alternatively, add a second source name to the for sou loop in step 2.


2. Using sma2casa.py

BE AWARE: This routine converts raw, uncalibrated SMA only. It is untested with newer versions of CASA and should not be considered reliable. A new routine for converting raw SMA data is under development.

Two python scripts are used for the conversion - sma2casa.py and smaImportfix.py. This section describes the most common use case, but a more detailed description, including a full list of options, can be found at sma2casa Details.

  • Retrieve scripts
    You can retrieve the python scripts from the github software repository.

    $ git clone https://github.com/kenyoung/sma2casa.git

  • Status
    The system temperature correction is applied by sma2casa.py. However, it is unable to create a Tsys table that is recognized as such by CASA. In CASA, you can still view the system temperature data and check the values.

    Warning: SMAImportFix.py only works with versions of CASA older than 4.7.

  • Instructions
    This sequence of instructions is tailored to RTDC machines running tcsh. The details will need to be modified to match your system.

    1. Check your current version of python.

      $ python --version

    2. If it isn't anaconda2, add the location to your path.

      $ setenv PATH /usr/local/anaconda2/bin/:$PATH

    3. Run sma2casa.py. The makevis.c amd makevis.so files will need to be in the same location as sma2casa.py. See sma2casa Details for a full description of the options.

      $ sma2casa.py 090202_07:19:01 -l -t

      sma2casa will produce a directory for each chunk and each sideband (e.g. Lower_s00, Lower_s01 ... Lower_s48), along with a file called sourceTable.

    4. Copy smaImportFix.py to your current working directory.

      $ cp /usr/bin/sma/smaImportFix.py.

    5. Run CASA (4.2).

      $ /opt/casapy-42.0.28322-021-1-64b/casapy

    6. Start to follow the steps outlined in this example script. The relevant line for calling smaImportFix.py is:

      CASA: execfile('smaImportFix.py')

      smaImportFix.py recognizes the presence of the Lower_sXX files created by sma2casa.py and creates a directory called MyDataLower (or MyDataUpper if dealing with the upper sideband).

      You may encounter FitsIDItoMS() warnings, or ones reporting "No valid pointing tables present", but these are benign and wont prevent the data being processed through to mapping.

    7. Read MyDataLower (for this example) into CASA and continue processing.

      CASA: vis ='MyDataLower'
      CASA: ms.open(vis)
      CASA: listobs()


CENTER FOR ASTROPHYSICS | HARVARD & SMITHSONIAN
60 GARDEN STREET, CAMBRIDGE, MA 02138