Overview

The RTDC archives and makes available data from the Submillimeter Array (SMA), the CfA Millimeter-wave Telescope, the Antarctic Submillimeter Telescope and Remote Observatory (AST/RO). We also provide hardware and software tools for the analysis of radio interferometric data from the SMA, JVLA, VLBA, and ALMA.

The RTDC is the home to SMA data and hosts the SMA data reduction webpages. Our internal users have access to all raw SMA data; whilst our online archives allow external users to search for a request public ans proprietary data as appropriate.

The CfA Millimeter-wave telescope archive includes the CO (1-0) spectra from the whole-Galaxy survey presented in Dame, Hartmann & Thaddeus (2001), v-l-b FITS cubes for each survey listed in Table 1 of that paper, plus larger composite cubes and integrated maps.

LEARN MORE | FAQs

Stay updated

Feb 7   SMA data has been reorganized on the RTDC. Find all pre-2020 data at /sma-arch/data/science/mir_data.20xx. 2020 data and newer continues to be found at /sma/data/science/mir_data.20xx. Flux and baseline data also follow this pattern.

Feb 6   wget2 is now available on the RTDC computers.

Feb 1   CASA 6.6.3 has been installed on the RTDC machines. This has been set as the default CASA version for our RHEL 8 machines rtdc10, rtdc11, rtdc12 and rtdc-sma-spec. For more details on this release see the CASA release notes. See Accessing CASA on the RTDC for full details of the available versions of CASA and python environments on the RTDC.

Jan 4   Hardware issues at the SMA have resulted in data being unable to transfer to Cambridge. We will notify when this is resolved. Update: Isuue is resolved. Data transfer is up to date.

Welcome to 2024   You can now find 2023 data in mir_data.2023.

Dec 8   The RTDC has added a new computer to its network - RTDC12. This new RHEL8 machine has 96 cores and 1.6TB of memory. You can find a list of all the RTDC computers here.

Oct 11   We are happy to anounce new look web archives for the SMA. Backed by a SQL database, the new look Public and Proprietary archives are faster, offer more search options, and allow users to request data from 200805 in CASA measurement set format in addition to raw MIR/IDL.

Oct 2   CASA 6.6 has been installed on the RTDC machines. This has been set as the default CASA version for our RHEL7 machines rtdc10 and rtdc11. For more details on this release see the CASA release notes. Make sure you have anaconda3 (python 3.8) set as your default python.

Sep 18   rtdc11 has had its RAM boosted from 384GB to 1.3TB, while rtdc9 has been boosted to 384GB.

Aug 9   Owing to an update to the data acquisition software, SMA users will notice new folder inside their data directories. The aux folder contains files that are supplemental to the main data, e.g. pointing reports and copies of scripts. The products folder is currently empty but will in future contain flagging/calibration tables.

May 14   Welcome to the new look RTDC webpages. If you have any requests for additional content please contact Holly.

May 11   The SMA Interferometry School starts next week here in Cambidge MA. Participants can find their data here.

Apr 27   CASA 6.5.5 has been installed on the RTDC machines. For more details on this release see the CASA release notes. This version will only work on rtdc10 and rtdc11. Make sure you have anaconda3 (python 3.8) set as your default python.

Mar 27  There is a delay getting data transferring from the summit to Cambridge. You can except to see data appear in the archive up to 24 hours after observations end. This is under investigation. FIXED.

Mar 1  Having problems with ssh after updating your MACOS to Ventura? Try adding these lines to your ~/.ssh/config file.
PubkeyAcceptedKeyTypes +ssh-rsa
HostkeyAlgorithms +ssh-rsa

Feb 8  Come and propose for SMA data. See SMA Science Observing Proposals for full details. The closing date is March 2 at 16:00 EST.

Feb 4  CASA 6.5.3 has been installed on the RTDC machines. This has been set as the default CASA version for our RHEL7 machines rtdc10 and rtdc11. A noteable highlight is a performance improvement for tclean. For more details on this release see the CASA release notes. Make sure you have anaconda3 (python 3.8) set as your default python.

Welcome to 2023   You can now find 2022 data in mir_data.2022.

7 Dec  The RTDC will be offline this morning for maintenace. BACK ONLINE NOW.

31 Oct  Interested in creating your own virtual python environment with Conda? See Running a Conda Virtual Envroment for instructions.

30 Sep  You can now find the python IDE PyCharm installed on the RTDC.

8 Aug  pyuvdata can be used to turn raw SMA data into a CASA MS. It is in the final stages of testing with an SMA-compatible release imminenet. Find out all about pyuvdata and how to use it here.

1 Jun  CASA 6.5 has been installed on the RTDC machines. For more details on this release see the CASA release notes. Make sure you have anaconda3 (python 3.8) set as your default python.

24 May  You can now find copies of historical SMAsh logs at /sma/logs. These can be used to extract detailed information regarding the priming setup of the array for a given observation.

11 May  Electrical issues in M342 have taken the RTDC machines offline. Repair is scheduled for later this week. FIXED.

12 Feb  Power distruptions to 160 Concord are affecting the RTDC network. This is under investigation. FIXED.

1 Feb  Check out the results of our speeds tests on Hydra. If you're interested in using Hydra for running parallel CASA tasks see the results at Comparing Disk Speeds on Hydra.

20 Jan  CASA 6.4 is now available on the RTDC. This is designed for RedHat 8 so users should run it on rtdc10 for optimal performance. You can find is at /opt/casa-6.4.0-16/bin/casa. See a full list of avalble versions of CASA at Accessing CASA.

Welcome to 2022   You can now find 2021 data in mir_data.2021.

22 Oct  Instructions for running the MIR task mir2ms have been added to Reading SMA Data into CASA. mir2ms uses IDL to convert raw SMA data into CASA measurement sets.

15 Oct  Find instructions for using pyuvdata to convert SMA data to UVFITS and CASA MS formats at Reading SMA Data into CASA. This is still in the testing phase but we encourage users to give it a try.

13 Aug  CASA 5.8 has been installed on the RTDC machines. It is aliased to casa. You can overwrite this in your .cshrc file if you want to use an older version (see Accessing CASA for a list). For more details on this release see the CASA release notes.

10 Jun  A new MIR update (JUNE 2021) is now available on sma-mir GitHub.

4 Jan  Remember you can now find data from 2020 in the mir_data.2020 directories.

2 Jan  CASA 5.7.2 is now aliased to casa on the RTDC. You can overwrite write this in your .cshrc file if you want to use an older version (see Accessing CASA for a list). For more details on this release see the CASA release notes.

14 Oct  The network between Hawaii and Cambridge is extremely slow at the moment. Expect delays with data being ingested into the archive. This is being investigated. UPDATE (11/12) The network has improved and we have parallelized the archive transfer script. However, to avoid overwhelming the summit network we are not transfering data to Cambridge until observations in Hawaii are complete (~8pm UT).

3 Sep  You can now find a comprehensive list of new SMA antennas files at Complete List of SMA Antennas Files. This list can be checked to see improved antenna positions exist for your data set.

20 Aug  CASA 5.7 is now available on the RTDC. This has been aliased to casa. You can overwrite this in your .cshrc file if you want to use an older version (see Accessing CASA for a list). For more details on this release see the CASA release notes.

15 Aug  CASA 5.8.0 pre-release is now available on the RTDC. Find it at /opt/casa-prerelease-5.8.0-5.el6.

17 Jun  SMA PIs will now find a download link for each data file listed in the observing report. This will take you straight to the given file in the proprietary SMA archive.

18 May  The public SMA archive now accepts a data directory name as a search option.

17 May  The proprietary SMA archive now reports more comprehensive information on your available data files. Any data from your project that has become public is now also listed.

1 Apr  rtdc7 now has 94GB of memory. Visit RTDC Computers to see details of all our machines.

19 Mar  Want to run a VNC session on the RTDC computers? Visit this page for directions.

17 Mar  Due to Covid-19, the SMA has suspended operations for 2 weeks.

19 Feb  If you want to learn more about interferometry, see the presentations from this years SMA Interferometry School here.

12 Feb rglinux13 has doubled its memeory to 96GB. Its raid array is performing optimally.

21 Jan  rtdc9 is experiencing problems. This will affect data retrieval from the SMA archive. Watch this space for updates. FIXED.

Welcome to 2021

9 Nov  Over the weekend we have seen a delay in getting new SMA data into the archive. This is due to network issues in Hilo and is being resolved.

2 Oct  The public SMA archive has a new look. You can now find supplemental plots showing environmental information, have the ability hide/show columns, and search for multiple sources or sets of coordinates.

2 Oct  CASA 5.6.1 is available on RTDC computers. This is now aliased to casa. You can overwrite this in your .cshrc file if you want to use an older version (see Accessing CASA for a list). CASA 5.6.1 contains new VLA and ALMA pipelines. For more details on this release see the CASA release notes.

12 Aug  CASA 5.6.0 is available on RTDC computers. This is now aliased to casa. You can over-write this in your .cshrc file if you want to use an older version. For more details on this release see the CASA docs page.

10 Jul   The SMA discovered a software error affecting Doppler tracking during science observations from 2011 April 4 to 2019 April 3. This error has a negligible effect on observations of continuum and broad spectral line, but narrow line observations are affected by a blurring of up to 0.8 km/s. There is a task in MIR for correcting this - find details here.

18 May  Access to the SMA archive internally through /sma/data/ is disabled while disks are reconfigured. Update: Access restored May 20.

14 May  CASA 5.5.0 is available on RTDC computers. This is now aliased to casa. Find a full list here if you need to use a previous version.

9 May  DS9 has been updated to version 8.0.

30 Mar  The archive now reports the receiver station (A or B) for each LO tuning. This had been potentially confusing when both receivers were tuned to the same LO frequency.

28 Feb  SMA PIs now get direct access to their data from the public and proprietary archives through new links in SMA observing reports (accessed through the SMAOC).

26 Feb  The proprietary archive results now list the sources included in each data file.

26 Feb  If you search by project code in the public archive, you will now find the results listed per data file rather than by source.

19 Feb  Follwing a bug fix CASA has been updated to 5.4.1-32. This is the version to use for the VLA pipeline. For the ALMA pipeline, version 5.4.0-70 has also been patched.

12 Feb  Warning: A bug has been found in SMARechunker. If a range of channels is selected the central velocity of the band is not properly recalculated. Check back here or at Reducing the Size of your Data for updates.

4 Feb  The antenna positions files required for correcting baselines are now publicly available for proprietary, (as well as non-proprietary), data. See Updating SMA Baselines.

7 Jan The SMA archive results list now includes the offset (in arsecs) between your search coordinates and the target coordinates of the returned observation. This only applies to a coordinates + radius search.

7 Jan  CASA 5.4.1 is available on RTDC computers. This is now aliased to casa. You can over-write this in your .cshrc file if you want to use an older version. See a full list here.

Welcome to 2020

12 Nov  We are having problems with rtdc9. Users cannot currently log on. This is being worked on. Update (13/11): Fixed.

30 Oct  The RTDC machines are now linked by a 10GB switch. This will speed up data transfer and make it faster to work on data stored on mounted disks. See a list of where to find mounted disks at Disk names and mount points.

18 Oct CASA 5.4.0 is available on the RTDC machines. See a full list of available versions of CASA here.

17 Sep  Version 11.5 of the VLBA/VLBI scheduling software SCHED is now available on the RTDC.

14 Sep  You can now search for polarimetry data via the new 'Polarization state' flag in the SMA archive. Be aware polarization data requires extra calibration steps. Email smarequester@cfa.harvard.edu for assistance.

27 Aug   The 31DEC18 version of AIPS is now available on rtdc7, rglinux12 and rglinux13. 31DEC16 remains the default on rglinux10.

1 Aug   See the results of testing parallelized tclean with CASA 5.3.0. There are some differences to the final map depending on the number of cores used, but on small scales the effect is mostly marginal.

11 Jul  You can now search by a specific frequency in the SMA data archive.

3 Jul   RADEX is available on RTDC machines.

11 Jun   CASA 5.3.0 is available on RTDC computers. This is now aliased to casa. You can over-write this in your .cshrc file if you want to use an older version. See a full list here.

5 Jun   The SMA archives are now up to date.

4 Jun   Network issues in Hilo has meant that recent SMA data has not been transferred to the RTDC. This affects the web-based archives as well as internal users. It will be posted here when the data becomes available.

22 May  Visit our new pages for running CASA in parallel: Testing Local Parallel Processing in CASA; Running CASA: Timing Tests; Instructions for running CASA in parallel

10 May  The virtual machines smadr1 and smadr2 have been renamed rglinux12vm and rtdc8vm. Visit RTDC Computers for more information.

8 May  You can now search the SMA archive by project code, and the columns can be sorted.

8 May  SMA data archive is up to date.

7 May  The weekend shutdown means the SMA archive is not currently up to date. Copying will begin once the Hilo machines recover from the earthquake power glitches.

7 May  The RTDC is back online.

4 May  RTDC computers are offline for scheduled power outage.

22 Apr  Many disk names, both local and exported, have changed. Find out if your disk is affected at Disk names and mount points

22 Apr  RTDC computers are back online. Power outage was postponed, so expect alternative weekend disruption.

20 Apr  RTDC computers will be switched off at 3pm for scheduled power outages.

06 Apr  Probable disk failure on rglinux11. Machine is unusable for the moment. UPDATE: disk repaired for now but is not considered reliable. Remember to back-up important data!

08 Mar  Unsure which machine is available? You can check the current status of the RTDC computers here. This information is updated every 15 minutes.

05 Mar  See the new instructions for importing uvfits files from MIR directly into CASA at Converting SMA data to CASA MS format.

01 Mar  RTDC computers back online.

28 Feb   The RTDC computers will be offline again tonight due to scheduled power outages.

28 Feb   The SMA science archive now accepts multiple sets of coordinates in a single search. Provide these in the 'RA Dec' selection box as a | separated list.

27 Feb  A new coordinates search option in the SMA Science Archive makes it easier to input coordinates.

25 Feb  RTDC computers back online as of 8pm.

23 Feb  RTDC computers shut-down for electrical work in 160 Concord.

23 Feb  The color printers psc342 and psc339 are now available from all RTDC machines.

22 Feb  OpenOffice can be started from the command line by typing openoffice. Aliases exist for the popular applications - oocalc, oowriter, and oodraw.

8 Feb The Karma package is available on RTDC machines. This can be used to make spectra, take 2 and 3D slices, display contours, and analyze expanding shells.

7 Feb  The SMA call for proposals for the 2018A semester is open. Find details here.

5 Feb  CASA 5.1.2 is available on RTDC computers. This is now aliased to casa. You can over-write this in your .cshrc file if you want to use an older version. See a full list here.

5 Feb  Updates to the web pages are happening today. Expect links to change. UPDATE: Changes are complete.

29 Jan  IDL 8.3 is now the default on RTDC machines. You can still access 6.2 by typing idl6.

29 Jan  See our expanded SMA data FAQ page. Topics include the archive, data format, and data reduction.

26 Jan  WIP is now working and exported to all RTDC machines.

17 Jan  Miriad-SMA-WB5.0.7 is now available on RTDC machines. In addition to fixes for several data files, the ephemeris tables are updated for solar system objects.

15 Jan  AIPS users are reminded to use rglinux10. Please report any issues.

Welcome to 2019

4 Dec  Miriad-SMA-WB5.0.6 is available on RTDC machines. This is the default when you source /home/miriad/miriad_WB/automiriad.csh. For pre-'4 chunk SWARM' data check here.

30 Nov  There is a new script to convert SMA data from MIR/IDL to UVFITS. Find out more at our updated instructions for opening SMA data with CASA.

19 Oct  Unsure which RTDC machine to use for data reduction? See Which machine should I use? for help.

16 Oct  A new version of Miriad (Miriad-SMA-WB5.0.5) is available. Known data issues are fixed, and a feature to record the information of mapping the frequency configurations between raw SMA data and Miriad is also implemented.

29 Sep  If the baseline solutions need updating, you will now find the correct antenna file provided along with the raw data in the science archive.

29 Sep  We bid farewell to our friend and colleague Shoshana Rosenthal who retires this week. We wish her all the best for the future.

19 Sep  Proprietary SMA data can now be accessed through our new Proprietary Archive. Available only to PIs or authorized users. You can also request your data be rebinned. More information can be found in the instructions.

14 Sep  CASA 5.1.0 is available. Find it here: /opt/casa-release-5.1.0-71.el6/bin/casa.

5 Sep  Use our new tool to easily perform unit conversions between frequency, wavelength and spectral resolution.

23 Aug  Looking for an easier way to view/filter catalog data? Try TOPCAT (Tool for OPerations on Catalogues And Tables). Available on RTDC machines.

10 Aug The SMA call for proposals is now open. Visit the SMA Observer Center for full details.

4 Aug  SMA science data is now at a new location. We are no longer archiving historical non-science data. See here for details.

20 Jul  CASA 5.0.0 is installed. Find it here: /opt/casa-release-5.0.0-218.el6/bin/casa. Make sure you are not running CASA on a mounted disk - this will slow it down considerably.

19 Jul  When you request SMA data through the Query Form, all sources contained in the raw data will now be listed.

3 Jul   Got questions about SMA data? Check out our new FAQ page.

27 Jun  CASA 4.7.2 is installed. Find it here: /opt/casa-release-4.7.2-el6/bin/casa.

21 Jun  See our updated instructions for converting SMA data to CASA MS format.

14 Jun  Summer students - new to Linux or want some new tricks? See our Linux tips page for a list of the most useful commands.

12 Jun  For RTDC users: an additional machine, rglinux8 (formerly mayet), is up and running in M-342.

1 Jun  A new archive is available here to search for calibration data (pointing/flux/baseline). External users will still have to visit the main Query Form to request the data for download.

12 May   For non-proprietary data (since 2007) you can view the observing report from the Query Form results list.

19 Apr  Archive search results now report the type of data that are available (raw, calibrated or imaged).

10 Apr  Priming data has been removed from the archive.

2 Apr  The RTDC computer list is now up to date.

20 Mar  All non-proprietary data is available for download from the SMA archive.

14 Mar  PI data now binned by a factor of 4 by default. This can be overridden if necessary by clicking a 'Full res.' checkbox on the PI request page.

8 Mar  CASA 4.7.1 installed. Find it here: /opt/casa-release-4.7.1-el6/bin/casa

19 Feb  Calibrated data is no longer offered as an option to PIs.

19 Feb  News-feed is added to the RTDC homepage. (2017)


Contact

Director   Dr Sean Andrews  |  IT/Archiving   Dr Holly Thomas
SMA Data Help    smarequester@cfa.harvard.edu