MPC Status Page

Known problems with contacting certain e-mail addresses are listed elsewhere.

Unless otherwise indicated, all times on the this page are local times (either Eastern Standard Time [UT-5h] or Eastern Daylight time [UT-4h], as appropriate).

Use the feedback form to report new problems or to comment on this page.

Status Messages and Outstanding Problems

• Unavailability of MPC webpages

  2008 May 14: 15:35. The MPC webpages hosted in the Computation Facility (as well as all CfA webpages so hosted) will unavailable for about 10 minutes starting at 18:15 today due to scheduled maintenance.

• Unavailability of MPC webpages

  2008 May 5: 18:30. The MPC webpages hosted in the Computation Facility (as well as all CfA webpages so hosted) were unavailable beginning at 17:30 due to disk upgrades. This work was announced but we missed the fact that it would impact webpage availability. The work has been completed.

• Upcoming Network Maintenance (April 24)

  2008 Apr. 17: 11:01. We have been informed that Harvard will be performing maintenance on one of its gateway machines on Thursday, April 24. The work will begin at 05:30 EDT and should be completed by 07:00. During this maintenance window there will may be intermittent connectivity into and out of the observatory.

• E-mail problems

  2008 Apr. 11: 15:00. As a result of a recent unannounced change in the e-mail system at CfA, there has been some disruption to our internal mailings over the past few days. There may also have been some disruption to outgoing e-mail. We have made the necessary changes to our e-mail setup and believe that all is back to normal.

• NEO Observations on MPEC 2008-G46

  2008 Apr. 9: 11:00. As a result of a designation snafu yesterday, some observations attributed to objects with K08G00x designations do not belong. These observations will be appearing on forthcoming MPECs under their intended designations.

• Mailing of Circulars

  2008 Mar. 29: 17:10. The machine used to mail out circulars (MPECs, IAUCs and CBETs), as well as Discovery Status reports, has apparently died. This machine is administered by the Computation Facility. They have been informed of the problem, but it is unlikely that they will look at the problem until Monday and if the machine has died it may be some time before a replacement is available. We will have to setup an alternate method of mailing out circulars.
  • 21:24. A partially-automated method of mailing out the circulars has been implemented. DOU MPECs will be delayed in their e-mailing.
    Subscribers who check sender addresses are warned that e-mails will not appear to come from the standard sources. Checks for e-mails should allow any address @cfa.harvard.edu. Failure to do modify the checks as mentioned here will cause you to miss circulars.
  • Mar. 31: 16:00. The immediate problem with the machine has been resolved. Automatic mailing of circulars has resumed.

• Webserver problem

  2008 Mar. 8: 12:00. A user has reported that "file locked" errors are still occurring when accessing web services. The AUTOACK problems, combined with staff absences, mean that it is unlikely that this problem can be looked at until next week.
  • Mar. 14. 12:00. Examination of the configuration of the new webserver showed one difference when compared to that of the old webserver. This change was made in the configuration and the webserver was restarted. We believe this will solve the problem: please let us know if this isn't the case!

• AUTOACK problem

  2008 Mar. 6: 17:20. A problem with DNS within our local network is causing mail sent to mpc@cfa to bounce back from the machine that processes incoming e-mail, because it thinks the message is to be relayed to another machine (which is forbidden). We are trying to get this fixed as soon as possible. This is something that we are dependent on our Computation Facility to fix.
  • 19:07. Not fixed as of now. Observers should continue to send observations as normal. Observation batches will be forwarded manually to the processing queues by MPC staff. Observers should ensure that every batch of observations has the AC2 keyword, otherwise they may not receive an ACK. Expect delays in ACKs overnight, as there will be no manual submitting between 23:30 and 09:30.
  • Mar. 7: 19:50. The DNS problem is still not fixed. It is unlikely that it will now be fixed before Monday. Expect delays in ACKs until at least Monday, with associated delays in returning designations to observers. Observers are asked to refrain from resending observations if they get bouncebacks containing:

            ----- The following addresses had permanent fatal errors -----
         autoack@ubasti
             (reason: 551 User not local, Relay disabled.)
             (expanded from: <MPC@cfa.harvard.edu>)
         

    If you get this error, we have your observations, you will be sent an ACK (albeit somewhat delayed) and the observations will be processed.
  • Mar. 8: 23:15. Since staff members also get copies of e-mails sent to mpc@cfa, I have just finished coding a routine to forward observation e-mails from my personal e-mail account to the automated routines. This routine is being run every 97 seconds, although it will not run between 04:00 and (possibly as late as) 05:00 EDT. As noted previously, you must use the AC2 keyword in order to receive ACKs. It is also worth remarking that the forwarding introduces some additional material into the message which will raise your "junk rating" score.
  • Mar. 10: 18:00. Still no resolution in sight.
  • Mar. 11: 00:00. It seems that the problem was fixed an hour or more ago, but we've had no confirmation from the Computation Facility. The special routine mentioned in the Mar. 8 report has been stopped. Normal ACK'ing should occur from now on.
  • Mar. 11: 16.00. The fix seems stable. It may be a day or so before internal processing returns to normal: priority will be given to processing observations.

• MPCORB.DAT file

  2008 Mar. 5: 09:00. The secondary datafile from which the MPCORB.DAT (and its varients) are produced was blanked out by last night's DOU MPEC process. Log files are being examined to see why this occurred. Once the reasons are understood, the file will have to be rebuilt.
  • Mar. 5: 11:15. The secondary datafile has been rebuilt. The publically-accessible MPCORB file will be restored on the next update cycle.
  • 12:00. The MPCORB files have been restored. The problem also affected, to a much lesser extent, the MPChecker files, and these are being rebuilt.

• Webserver issues II

  2008 Feb. 26: 10:00. A number of observers have reported erroneous junk rating values in their acknowldegement messages. This problem is being investigated.
  • Feb. 27: 10:00. This problem is believed to be fixed.

• Webserver issues

  2008 Feb. 25: 11:10. A number of users have reported (intermittent) problems with "file protection" messages appearing when using web services. There appear to have been some issues with protections on directories when the webserver was moved over to the new machine. The reported permanent problems appear to have been resolved.

• Webserver move (Feb. 19)

  2008 Feb. 14: 15:35. The MPC webserver will be moving to a new, faster computer. The current webserver will need to be shutdown at noon, to allow a clean copy of numerous files to the new machine. AUTOACK will also be disabled at the same time. It may take several hours to transfer all the necessary files. Once the new webserver is restarted users should see no difference in accessing web services. There may be delays in propagating the necessary DNS change--this is out of our control.
  • Feb. 19: 10:40. The automated processing queues have been stopped.
  • 11:47. The AUTOACK process has been stopped.
  • 12:00. The webserver has been stopped.
  • 14:10. All files appear to have been to new locations. Waiting for local DNS to update before restarting webserver.
  • 15:00. AUTOACK process has been restarted.
  • 15:30. Automated processing queues have been restarted.
  • 18:00. Webserver still off-line as required DNS changes not yet made.
  • Feb. 20: 10:48. Have restarted the old webserver as no sign of DNS changes being made.
  • 13:30. DNS changes have been made, visible internally. Apparently change was visible externally around 12:00.

• CF mail server outage (Feb. 21)

  2008 Feb. 14: 15:31. We have been informed that the CF's SMTP server will be rebooted at 07:00 EDT on Thursday, Feb 21. During the expected 30-minute outage, incoming mail (e.g., to mpc@cfa) will not be received, but the mail should be resent by the remote systems when the SMTP server is restarted.

• Galaxy names in SNChecker

  2008 Feb. 6: 08:39. A user reported that the SNChecker was apparently not resolving galaxy names correctly. A check showed this to be the case and a modified executable was put on-line.

• Missing Objects in MPChecker

  2008 Feb. 4: 14:39. A user reported that an object was missing from the output of MPChecker. An investigation showed that the file of orbits used by the MPChecker service had not been built correctly by the overnight procedure due to two processes trying to write to the same file. This problem was believed fixed some months back. It is now fixed.

• DOU MPEC

  2008 Jan. 19: 14:00. Last night's DOU MPEC failed due to a malformed orbits in one of the files crashing one of the check programs. The check program has been recoded to prevent a similar occurence in the future from crashing the program. The offending orbit has been removed and the bits of the DOU MPEC that failed are being rerun.

• Special-Epoch NEA*.DAT Files

  2008 Jan. 12: 22:00. A user reported that the elements for 2008 AF₃ in the special-epoch NEA*.DAT files seemed incorrect. Investigation showed that the same unchanged elements are used in all the NEA*.DAT files. The same erroneous behavior occurs for a number of other objects. The reasons for this problem are as yet unclear. It is possible that it is related to the Dec. 12 problem listed below. Further investigation will be undertaken after the update tonight associated with the DOU MPEC. Until this problem is resolved, users requiring NEA elements for non-standard epochs are advised to use the MPES to obtain this information.
  • Jan. 13: 17:35. A problem with a bit of rarely-executed code in the program that generates the NEA*DAT files has been located and, we believe, fixed. The update associated with tonight's DOU MPEC should remove the problem with some of the elements in the NEA*.DAT files. This problem affected only the NEA*.DAT files, not any primary orbit files.
  • Jan. 14: 06:20. A new set of NEA*.DAT files were put on-line around 02:30. The new set is based on orbits from the previous DOU MPEC: a set based on orbits from the current DOU MPEC is currently being run.
  • Jan. 14: 08:40. The NEA*.DAT files are now based on orbits from the latest DOU MPEC.

• Recently-Designated Vaisala Orbits in MPES

  2007 Dec. 20: 10:30. A user reported that the MPES was not returning ephemerides for a number of objects designated since MPC preparation started. These objects have only Vaisala orbits and the rebuilding of some MPES orbit files during the MPC preparation process has caused these orbits to become unavailable to the MPES. The resumption of MPC processing mode in a few hours should make these objects available again.

• DOU MPEC

  2007 Dec. 17: 07:55. Last night's DOU MPEC failed due to two malformed orbits in one of the files. The offending orbits has been replaced and the bits of the DOU MPEC that failed are being rerun.

• NEA*.DAT files

  2007 Dec. 12: 10:50. The NEA*.DAT files available on the ftp site are truncated due to a corruption of a secondary data file used by the process that generates those files. The corrupted file has been replaced and the files should be corrected after the next DOU MPEC is issued.

• DOU MPEC

  2007 Dec. 6: 10:50. Last night's DOU MPEC failed due to a malformed orbit in one of the files. The offending orbit has been replaced and the bits of the DOU MPEC that failed are being rerun.

• Access Problems?

  2007 Dec. 6: 10:40. A number of e-mail reports have asked if we have problems with our cgi server. There are no problems internally, the webserver is responding to requests normally and quickly. Any access problems are caused by something between the MPC and the end-user. Our Computation Facility reports that there was a nine-hour outage yesterday at an observatory site about two miles from the main site, but they gave no indication that this outage affected the whole observatory.

• Apparent Problem with AUTOACK TEL Lines

  2007 Nov. 30: 21:00. A recent change in MPC preparation has required us to preprocess TEL lines when the batches are ACKed. A side-effect of this is that valid TEL lines are erroneously being flagged as incorrect. A new executable that removes this erroneous warning is now in place.

• Possible Service Interruptions Nov. 28-30

  2007 Nov. 28. Today we are starting some work on our cluster machines. We will bringing a new machine on-line (which will become the new webserver machine), configuring external shadow-set disk-storage devices and upgrading the old machines from VMS V8.2 to V8.3. This work will take place over the next several days. While the current webserver machine is being upgraded there will obviously be an interruption in cgi service. There may also be brief periods when cgi services are unavailable while other cluster machines are being upgraded.
  • Nov. 29: 10:30. The old cluster machine upgrades are starting.
  • 15:10. The webserver upgrade is starting.
  • 16:45. The upgrades are finished for the day. Automated processes, such as AUTOACK, are being restarted.
  • Nov. 30: 08:35. The automated processes, including AUTOACK, have been paused, to allow the upgrades to continue.
  • 18:25. The upgrades have been completed. Automated processes, including AUTOACK are being restarted. Configuration of the new webserver will begin over the weekend, but transfer of web services from the old webserver may take some weeks (a lot of data needs to be shifted and the move needs to be coordinated with the Computation Facility). Transfer of data directories to the new shadow sets will also begin this weekend, but this should have zero impact on internal and external services.

• Display Circular scripts reactivated

  2007 Oct. 31: 17:00. The Display Circular scripts found on the pages of recent MPECs, IAUCs and CBETs have been reactivated. The Display Circular scripts on the pages of returned circulars are NOT reactivated, neither are the Display PS/PDF options on the IAUC circular pages. We are considering how best to update the static HTML pages with the outdated script calls and restore the missing functionality.

• Webserver rebooted

  2007 Oct. 17: 11:15. The MPC webserver rebooted itself automatically this morning. Normal service should have resumed.

• Webserver sluggish

  2007 Oct. 16: 13:45. The MPC webserver is sluggish, as a result of a large number of running jobs. We hope to have a new, faster webserver on-line within a month.

• Network Problem?

  2007 Oct. 13: 10:07. A problem has caused five computers of the cluster to reboot themselves just before 03:00. This is the same situation that occured on Oct. 5. AUTOACK was stalled until a few minutes ago. We believe normal service has been restored.

• Network Problem?

  2007 Oct. 5: 21:17. A problem has caused five computers of the cluster to reboot themselves. Since all are located in the same building (which is different to all the other cluster members) and all are on UPSes, it was presumably a network outage of some type. We believe that normal services were restored within minutes.

• Two bug fixes

  2007 Sept. 10: 08:30. Two long-standing bugs in services have been fixed. A problem affecting R.A. ordering of NEOCP objects at certain times of the year has been fixed, as has a problem with the non-appearance of certain comets (those with perturbed orbits published after the last MPC batch) not appearing in the MPChecker services.

• Missing Observations in MPCOBS

  2007 Sept. 3: 09:31. A user reported that observations of a specific object published in the most recent mid-month MPS batch were not available in the MPCOBS service. An examination of the report seems to show that all the observations from that batch are unavailable. We are investigating.
  • 10:02. The problem has been fixed. The mid-month MPS observations are available in MPCOBS.

• Missing Orbits in MPES

  2007 Aug. 14: 11:26. A problem with the overwriting of one of the main pending orbit files with some old elements has caused some elements to be unavailable in the MPES. We are rebuilding the affected file.
  • 12:36. The file has been rebuilt. All the elements in this file will be republished on the next DOU MPEC.

• Upcoming Network Maintenance (August 9)

  2007 Aug. 8: 11:23. We have been informed that Harvard will be performing maintenance on one of its gateway machines on Thursday, August 9. The work will begin at 05:30 EDT and should be completed by 07:00. During this maintenance window there will be a loss of connectivity into and out of the observatory.

• Power Glitch at Observatory

  2007 Aug. 7: 12:37. A glitch in the electricity supply at the Observatory appears to have fried one of our UPSs, with the result that three machines and some networking equipment lost power. We have moved equipment around, restarted the affected machines and everything seems to be back to normal.

• Mid-month Observation Files in ECS

  2007 Aug. 5: 10:21. Due to an apparent problem in shifting one set of cronjobs from a to-be-decommissioned CF linux box to a new linux box, the files of complete through the latest mid-month batch of MPS seem to have been built incorrectly. We have checked the logs from the VMS end and see no problem. We have no logs on the linux side where the problem occurred, so diagnosis may be difficult. We hope that the problem will be identified and fixed by the next mid-month MPS batch.
  • 10:47. Further checking shows that all the observations of numbered minor planets appear to be in the midmonth/mpn.arc file, but they are not sorted correctly. The midmonth/mpu.arc appears to be complete and sorted correctly.

• Checker files

  2007 Aug. 1: 14:00. Due to a problem filing discovery information during the preparation of the July 30 MPCs, a number of secondary datafiles were built incorrectly, including some used by the MPES. We are rebuilding the affected datafiles.
  • 14:21. We believe all affected files have been rebuilt.

• Checker files

  2007 July 26: 14:47. Following a Feedback comment, it has been determined that the procedure that updates the odd-epoch elements for use in the various checker services has been failing in recent days due to a diskspace problem. The services are functioning, but are simply missing recent orbits. We are attempting to restore the missing orbits.
  • 15:15. We think we have restored the missing orbits. The two cases that were mentioned in the Feedback comment seem to have been corrected.

• Comet magnitudes in MPES

  2007 July 24: 20:50. A problem that caused comet total (m1) magnitudes to be output when fainter than the nuclear (m2) magnitudes has been fixed.

• Minor Planet Designation script

  2007 July 24: 11:54. A new version of the cgi script that is executed when one selects "Display all designations for this object" in the MPES has been installed (see the Mar. 30 status message). A version that will work on the Minor Planet Designations page will be forthcoming (it has to cope with a much wider range of possible inputs than the MPES version).

• Upcoming Network Maintenance (July 26)

  2007 July 23: 12:22. We have been informed that Harvard will be performing maintenance on one of its gateway machines on Thursday, July 26. The work will begin at 04:30 EDT and should be completed by 06:00. During this maintenance window there will be a loss of connectivity into and out of the observatory.

• Mis-flagged objects for numbering

  2007 July 1: 18:43. A number of objects are being erroneously indicated as being "flagged for numbering in the next full MPC batch", principally a couple of one-opposition NEAs! This problem may correct itself after tonight's DOU MPEC is issued.

• Archive of older enhancements and problems/resolutions

MPC homepage