Friday, August 1, 2014

Calculating the uncertainty of a photometric calibration from standard stars

I've been working on lots of things lately and haven't been taking  the time to post.  Today's task is obtaining calibrated optical magnitudes for brown dwarfs that I've been monitoring with relative photometry in an f814w filter with a bandpass from about 700-950nm.  I have already finished analyzing and interpreting the relative photometry, which was my main scientific objective since this type of photometry offers the best sensitivity for detecting new variable objects -- but the calibrated magnitudes are also of interest, especially because for many of the objects I measured, no published optical magnitudes exist.

The first step is to obtain a photometric calibration: that is, a means of converting the total number of ADUs measured for a given star on a given image (e.g., using aperture photometry) into a calibrated magnitude for the star.  For a simple calibration, only a single constant is necessary.  For example, the magnitude of a star is:

m = zeropoint - 2.5*log(ADU/t)

Where the log is base 10; t is the exposure time in seconds; and zeropoint, the single constant of the calibration, is the photometric zero point magnitude: that is, the magnitude a star would have if it produced a flux of only 1 ADU/sec with the telescope, detector, and filter being calibrated.  Note that in most observing situations, objects with a flux of 1 ADU/sec are well below the detection threshold: hence, one cannot actually image an object whose brightness equals the zeropoint magnitude.  Because I don't like this property, I prefer to use a different constant in my calibrations: the ADU/sec for a tenth magnitude star, or TMCS (Tenth Magnitude Counts per Second).  The magnitude formula is then:

m = 10.0 - 2.5*log(ADU/(t*TMCS))

Suppose then that I have several images (all in the same filter) of a calibration field containing several standard stars of known magnitudes.  From each star on each image I can calculate a photometric calibration (whether I choose to use the zero point or TMCS is unimportant).  The interesting question is how to average together the different calibrations to find a final, master calibration -- and how to calculate the uncertainty of this final calibration.

If uncertainties on the individual magnitudes of the stars are available, one can take them at face value and proceed as follows:

1. Find the mean and standard error of the ADUs measured for each given star across the different images.

2. Use these values to calculate a separate photometric calibration for each standard star, including an uncertainty estimate that takes into account the contributions from the published uncertainty on the magnitude and the standard error of the measured ADUs.

3. To arrive at the final calibration, take a mean over the calibrations obtained for each individual star, weighted by the inverse squares of the uncertainties calculated in the previous step.  Calculate the uncertainty of this final calibration accordingly.

I could not adopt this method with all of my standard star fields, however, because in some of them the published magnitudes of the stars did not have uncertainties.  Even where uncertainties exist, I prefer not to take them at face value, since it is difficult to be sure than any given standard star is not a low-level variable, in which case the published uncertainty is likely to be an underestimate.

I could use a very simple method: simply find the unweighted mean and the standard error over all of the individual calibration measurements, with no prior averaging over any star or image.  However, this could produce very erroneous results in the following scenario:  Suppose there are only two stars standard stars, but one of them is an unknown variable that has changed brightness markedly since it was identified as a standard.  The calibrations derived from the two stars are therefore quite inconsistent, and there is no way to tell which is closest to the true value.  Suppose further that there are many images: say, 18.  Thus the standard error will consist of the standard deviation of 36 measurements divided by the square root of 36.  Thus it will be one-sixth the standard deviation.  However, the uncertainty on the calibration is really about half the difference between the mean calibrations from each star, which will be comparable to the standard deviation: six times larger than the standard error.  Thus the error will be severely underestimated.

My way of resolving this problem and producing a final uncertainty measurement with contributions from both the uncertainties of the stellar magnitudes and those of the ADU measurements on the images was, I think rather clever.  Here is how it works:

1.  Calculate an independent calibration from each star on each image.

2.  For each individual star, find the mean and standard error of its calibration across all images.

3.  Construct a weighted average of these single-star calibrations to arrive at a final calibration and uncertainty.

Thus far I have obtained a good calibration, which weights bright stars with consistent measurements on my images most strongly.  However, its uncertainty is underestimated because it takes into account only the scatter in measurements on my images, not the uncertainties of the published magnitudes that were input to the calculation.  I proceed to estimate this contribution and then calculate the final uncertainty as follows:

4.  For each individual image, find the mean and standard error of the calibrations obtained for all stars in that image.

5.  Find the average of these individual-image standard errors.

6.  Add (in quadrature, of course) this average standard error to the previously calculated uncertainty on the final calibration.

This method will produce an accurate uncertainty in the case noted above of two stars yielding discrepant calibrations.  Its only disadvantage is that in calculating the individual-image standard errors and folding in their average value to the final uncertainty, it effectively double-counts the random noise in the measurements from calibration images.  Thus, where the published magnitudes of the standard stars are highly precise and accurate, the final uncertainty will be overestimated.  However, in most cases the measurement noise will be less than the true uncertainty in the published magnitudes, so this overestimation of the uncertainty will not be severe.  Furthermore, it is usually much better to overestimate than to underestimate the uncertainty of one's measurements, since the latter can lead to serious scientific misinterpretation of results that are in fact due to nothing more than measurement errors.

As a final note: a good photometric calibration, for ground based data, should also include a determination of how the measured flux varies with airmass.  A linear fit with associated uncertainty is usually sufficient.

No comments:

Post a Comment