Making a Sky Background Template
Making a Sky Background Template Eventually, you're likely to want to subtract either a fringe frame (i' and z' filters) or a sky template (u'g'r') from your exposures, so that off-axis scattered light and/or fringes don't appear in your final mosaic. This page describes the steps needed to create and remove such background templates from the individual exposures. Note: Megacam headers use the ZPN coordinate projection. As of this update, IRAF doesn't understand ZPN, so you will delay running the zpn task until just prior to converting these multi-extension FITS files to single-extension files (step VII.C).
I used crutil to make CR hit masks first, then nproto/objmask to make object masks, and then, once
the masks had been verified and applied, I used sflatcombine
to create a sky template, as described very nicely at the
NOAO/MOSAIC page.
The specific steps are as follows, starting with some parameter setup and then the cosmic-ray
removal:
np> set masktype = pl
np> ls EGS*.fits > om.obj.list
np> vi om.obj.list # (to change ".fits" --> "om" on all lines)
For u', g', and r', also be sure to set
For i' and z', hsigma=3.0 caused much of the actual fringing structure to be flagged in the object masks. So when fringes are present one must use
In either case, one then makes an object-masked average-stacked frame with sigma clipping to eliminate the cosmic ray hits not cleaned out by crutil. I haven't come up with a perfect solution for this yet, but the following will reliably correct all the single-pixel hits (note it assumes the file names begin with EGS, which is probably not true for you):
cr> mscstat EGS*.fits fields="image" format- > cravg_script
cr> cp cravg_script xx
cr> vi xx # vi commands follow
:1,$ s/\]/"/
:1,$ s/.fits./_/
:1,$ s/EGS.\.//
:wq # save and quit vi
cr> vi cravg_script # vi commands follow
:1,$ s/E/craverage E/
:1,$ s/\]/\] output="" crmask="crmask/
:wq
cr> !paste cravg_script xx > xy
cr> vi xy # to get rid of the hidden tab character
cr> mv xy cravg_script
cr> del xx
The above is my klugey way of making instructions like the following -- there certainly must be better ways of doing it; from this partial listing (for a run covering the IRAC deep field) you get the idea of what's needed. This will dump all the crmasks into the current working directory:
craverage IRACCF.0373.fits[IM1] output="" crmask="crmask_0373_IM1"
craverage IRACCF.0373.fits[IM2] output="" crmask="crmask_0373_IM2"
craverage IRACCF.0373.fits[IM3] output="" crmask="crmask_0373_IM3"
craverage IRACCF.0373.fits[IM4] output="" crmask="crmask_0373_IM4"
craverage IRACCF.0373.fits[IM5] output="" crmask="crmask_0373_IM5"
craverage IRACCF.0373.fits[IM6] output="" crmask="crmask_0373_IM6"
craverage IRACCF.0373.fits[IM7] output="" crmask="crmask_0373_IM7"
craverage IRACCF.0373.fits[IM8] output="" crmask="crmask_0373_IM8"
craverage IRACCF.0373.fits[IM9] output="" crmask="crmask_0373_IM9"
craverage IRACCF.0373.fits[IM10] output="" crmask="crmask_0373_IM10"
craverage IRACCF.0373.fits[IM11] output="" crmask="crmask_0373_IM11"
craverage IRACCF.0373.fits[IM12] output="" crmask="crmask_0373_IM12"
Now, run the cosmic-ray removal script:
cr> bye
Now that the cosmic ray masks have been generated, they are applied to the images. First the BPM header parameters must be changed to point to the new cosmic ray masks, and then fixpix is run to perform the actual masking. It's extremely wise to blink a couple of masks against the data before applying them!
This procedure is quite good at getting almost all the significant single-pixel cosmic ray hits, and a great many of the relatively rare multiple-pixel cosmic rays. These occured at a rate of 1-2 per second per chip in the 2006 September Megacam images, in other words with long exposures they accumulate rapidly.
This algorithm falls down when a really energetic hit corrupts a dozen or more contiguous pixels, or when a low-level hit falls atop a real source. In my experience such cases are rare, and get diluted by coadding so are not terribly difficult to tolerate.
cr> vi bpm_into_headers.macro
:1,$ s/craverage/hedit/
:1,$ s/output.../BPM/
:1,$ s/crmask="//
:1,$ s/"/.pl add+ update+ verify- delete-
:wq
cr> cl < bpm_into_headers.macro
cr> mscstat EGS*.fits fields="image" format- > fixpix_in
cr> fixpix @fixpix_in BPM
Next step: create the object masks so that the sky templates are uncontaminated.
np> objmask.masks=""
np> objmask EGS*.fits objmasks="@om.obj.list"
np> bye
me> sflatcombine.combine="average"
me> sflatcombine.reject="sigclip"
me> sflatcombine.mclip=yes
me> sflatcombine.output="Sflat"
me> sflatcombine.scale="mode"
me> sflatcombine EGS*.fits combine="average" scale="mode" masktype = "!OBJMASK" output="Sflat" reject="sigclip" mclip=yes
Lastly, run Brian's least-squares fringe removal program. This is done from a Linux prompt and not within IRAF. The syntax is:
...where "X" of course is a place holder for the specific characters needed to flesh out the file names. Finally, inspect the output EGS*skysub.fits files individually. Some more intervention may be required. For the January 2006 run using SDSS z' data with strong fringeing, I found that chips im5, im7, im9, im65, and im67 had some kind of uncorrectible background problem so I summarily deleted only those extensions from the sky-subtracted data as follows:
Note that there is likley NO REASON TO DO THIS for any other bands apart from z'! One must use fxdelete because simply setting all the pixel values to -10000.0 fails: swarp will put zero-valued pixels in the output files.
If, upon inspecting the skyflat image you find it free of artifacts, you are done with this step. Note that for i' and z' reductions, it is usually the case that this procedure has to be iterated at least once, using the Nth-iteration sky-subtracted frames as input for iteration N+1. Iterating allows significantly better object masking, as the fringes are reduced by 90% after a single iteration.
Last updated July 2, 2008.