Threaded Mode | Linear Mode

Mischa Schirmer · 07-27-2006, 18:40

Hi Emmanuel,

In case of excellent seeing, which is often encountered in the near-IR, typical imagers start to undersample the PSF very strongly. That usually leads to the effect that Eye cannot distinguish anymore safely between cosmics and the dense cores of the stellar PSF. I tried relaxing the corresponding Eye parameter, which subsequently preserves the PSF, but also the cosmics.
Thus, I wondered if you could implement in the final Swarp step - when the resampled images are stacked - a filter that simply throws away the highest pixel from the stack.
Of course one can think of a more sophisticated approach (iterative outlier rejection or the like), but a simple one would certainly help already very much.

Thank you!

mischa

***Emmanuel Bertin*** · 07-28-2006, 13:47

I am not sure to understand. How would other values be managed? As a WEIGHTED average? Shouldn't COMBINE_TYPE MEDIAN do the job? I am afraid that without proper clipping such an estimator would significantly bias the output.
I earlier thought of a weighted sum of all pixel values except the lowest and highest ones; but in most situations the gain with respect to a simple median should be negligible.
What do you think?
Emmanuel.

Mischa Schirmer · 07-28-2006, 14:27

In our pipeline (THELI) we use weight maps which are based on the normalised flat field, i.e. they have not only weights of 1 and 0 but everything in between. I assume they are simply neglected when using the COMBINE_TYPE MEDIAN combination instead of the WEIGHTED option? Thus the optimal S/N of the coadded image would be lowered, which we try to avoid.

Indeed the MEDIAN combination would throw away the cosmic, but if the number of images is small, the cosmic would bias the unfiltered median towards a higher value.

What I have currently in place is a programme that I run after the input images have been resampled. It looks up all the resampled input pixels for a particular output pixel in the coadded image, detects any outliers, and sets the corresponding pixels in the previously resampled weight map to zero. I then combine using the WEIGHTED average method. However, that approach is rather inefficient for a larger number of exposures (likely due to my inferior programming), and I wondered if it was a simpler solution implementing it directly in Swarp since all the transformations etc are known there already.

Best regards!

Mischa Schirmer

***Emmanuel Bertin*** · 07-29-2006, 20:27

Quote:In our pipeline (THELI) we use weight maps which are based on the normalised flat field, i.e. they have not only weights of 1 and 0 but everything in between. I assume they are simply neglected when using the COMBINE_TYPE MEDIAN combination instead of the WEIGHTED option?

True.

Quote:Indeed the MEDIAN combination would throw away the cosmic, but if the number of images is small, the cosmic would bias the unfiltered median towards a higher value.

Well hopefully the fraction of pixels hit by cosmic rays is fairly small. I believe it is better to bias only those ones than the vast majority that does not suffer from cosmic rays!

Quote:What I have currently in place is a programme that I run after the input images have been resampled. It looks up all the resampled input pixels for a particular output pixel in the coadded image, detects any outliers, and sets the corresponding pixels in the previously resampled weight map to zero. I then combine using the WEIGHTED average method. However, that approach is rather inefficient for a larger number of exposures (likely due to my inferior programming), and I wondered if it was a simpler solution implementing it directly in Swarp since all the transformations etc are known there already.

OK what you want is basically a weighted average with rejection. Let me see what I can do, and I will come back to you!
Emmanuel.

elinor_lev · 01-05-2011, 22:43

Hi,

I see this is an old thread, but I suffer from the same problem of cosmic rays. Usually I combine with sigma-clipping rejection, which gives better results than median.
Any chance of implementing this in SWarp any time soon?

Thanks,
Elinor

(07-29-2006 20:27)Emmanuel Bertin Wrote:
Quote:In our pipeline (THELI) we use weight maps which are based on the normalised flat field, i.e. they have not only weights of 1 and 0 but everything in between. I assume they are simply neglected when using the COMBINE_TYPE MEDIAN combination instead of the WEIGHTED option?
True.

Quote:Indeed the MEDIAN combination would throw away the cosmic, but if the number of images is small, the cosmic would bias the unfiltered median towards a higher value.
Well hopefully the fraction of pixels hit by cosmic rays is fairly small. I believe it is better to bias only those ones than the vast majority that does not suffer from cosmic rays!

Quote:What I have currently in place is a programme that I run after the input images have been resampled. It looks up all the resampled input pixels for a particular output pixel in the coadded image, detects any outliers, and sets the corresponding pixels in the previously resampled weight map to zero. I then combine using the WEIGHTED average method. However, that approach is rather inefficient for a larger number of exposures (likely due to my inferior programming), and I wondered if it was a simpler solution implementing it directly in Swarp since all the transformations etc are known there already.
OK what you want is basically a weighted average with rejection. Let me see what I can do, and I will come back to you!
Emmanuel.

Mischa Schirmer · 03-23-2011, 23:40

Hi Elinor,

try THELI. Our pipeline is largely based on Astromatic tools and the outlier rejection filter is fully implemented for the coaddition.

http://www.astro.uni-bonn.de/~theli/gui/

mischa

(01-05-2011 22:43)elinor_lev Wrote: I see this is an old thread, but I suffer from the same problem of cosmic rays. Usually I combine with sigma-clipping rejection, which gives better results than median.
Any chance of implementing this in SWarp any time soon?

Thanks,
Elinor

bhg · (This post was last modified: 04-11-2011 18:02 by bhg.)

Hello Emmanuel,

Indeed, it would be nice if SWarp could at the same time combine images and reject outliers (cosmic rays, leftover bad pixels, etc.) using a rejection algorithm like minmax (cf IRAF). In fact, preparing bad pixel masks (as input weight-maps) is a very tedious task, especially for large detectors where bad pixels/artifacts are not always neatly recovered. Furthermore, the interpolation resampling usually increases the size of bad pixels regions, which is not nice at all. When numerous images are available for an average- or weight-combination and a preliminary pixel interpolation has been applied to remove the most obvious bad pixels (e.g. bad pixel columns or lines), it is then very convenient to reject about 5-10% of the values, corresponding to the remaining defects (cosmic rays, leftover bad pixels, etc.).

Thanks,
Gabriel