Note - the data for all the interactive sessions is available for download.

Hands-On Exercise 4:

Searching for a Nova in PTF Data

Version 0.1

The goal of this exercise is to find a nova in archival PTF images. (As a quick aside - note that the process of searching for novae and supernovae [SNe] is essentially identical. PTF searches for both using the same methodology.) Recall from the previous talk that novae are nuclear runaway events on the surface of an accreting white dwarf that is in a tight binary system. These explosions lead to bright transients, which allows us to study novae in both the Milky Way and nearby galaxies.

The optimal method for searching for novae (and SNe) is to use image subtraction (see the Image Subtraction talk by Peter Nugent). The end of this exercise has a black belt level question where you are asked to perform image subtraction. For the main portion of this exercise, however, we will search for novae using photometric catalogs. While this method is not as complete as image subtraction, it is still possible to discover many explosive transients this way.

Recall that PTF source catalogs are produced by SExtractor, astronomical source detection software that constructs catalogs of objects detected in astronomical images. We will begin our search by looking for novae in the Andromeda Galaxy (M31), a Milky Way "twin" located less than 1 Mpc away. For simplicity, we will focus on a single PTF field and detector for the present exercise.

By Y Cao (c) 2014 Aug 22

Problem 1) Searching for Novae via a catalog cross match

To begin our catalog search we need a reference image -- typically this is a deep stack of images taken prior to the epoch on which we wish to search for transients. When a new image is obtained, any new transients will be detected in that image, but they should not be present in the deep reference image.

For this problem, we refer to the reference as epoch0. The corresponding data are found in \(\texttt{data/M31/epoch0/}\), the \(\texttt{SExtractor}\) catlog is named


and the actual image of the field is named


As a point of comparison, we will examine files from \(\texttt{data/M31/epoch1/}\), which were taken after the images from epoch0. The epoch1 catalog is named:


and the image of this field is named:


The catalogs are fits data tables, and the images are fits images. For the cross match between the catalogs, we will need the positions of all the sources that are detected. In these catalogs the Right Ascention (\(\alpha_{J2000}\)) is given by the keyword \(\texttt{X_WORLD}\), while the Declination (\(\delta_{J2000}\)) is given by the keyword \(\texttt{Y_WORLD}\). Note - these keywords are different from the ones we used for the How to Construct Light Curves from PTF Observations exercise.

In a typical PTF image, the astrometric solution is good to \(< 0.5 \textrm{arcsec}\), while the typical seeing for the P48 telescope is \(\sim 2 \textrm{arcsec}\). Thus, here and for all portions of this exercise, we recommend using a \(1 \textrm{arcsec}\) match radius to compare sources in different catalogs.

Part A) Cross match new image catalog to the reference

Using the reference catalog, and the sources detected in epoch 1 -- identify candidate novae by determining which sources present in epoch 1 are not present in the reference. How many candidate novae did you find?

Hint 1 -- remember that distances in this case are measured on a sphere, and that conversions between radians, degrees, and arcsec all matter.

Hint 2 -- your list will be fairly large.

Hint 3 -- if you are completely flumoxed, check out the code from How to Construct Light Curves from PTF Observations to get an idea of one way to write a cross-match code.

In []:
### cross match sources in the ref and epoch1

refcat = # COMPLETE
epoch1cat = # COMPLETE

Part B) Eliminate Some of the False Positives

As noted above - your list of candidates should contain a large number of candidates. Many of these candidates are false positives (e.g. asteroids, cosmic rays, etc.). One way that PTF mitigates against these types of false positives is to take $\(2 images of every field observed in a given night. With images that are separated by \)1 $, and asteroids will have moved, and any cosmic rays are unlikely to land on the same location twice (this can happen, however, so one must use caution even with two images).

Fortunately, novae (and SNe) are long lived (\(\sim \textrm{days} - \textrm{months}\)), so if you have found a genuine transient, it should be present in both images taken on the same night.

Eliminate some of the false positives in your original list of candidates by excluding those sources that are not detected in the second image. The data for epoch2 can be found here: \(\texttt{data/M31/epoch2/}\).

Hint 1 - bookkeeping is a major challenge when dealing with large surveys with many images, and millions or billions of total detections. Be careful to keep track of your variables, images, etc.

Hint 2 - it might be easiest to find sources in epoch 2 that are not in the reference, using the same code you wrote above. Then compare the candidates in epoch 1 to candidates in epoch 2.

In []:
### make a master list of candidates by identifying sources
### in epoch1 and epoch2, but NOT in the ref


Congratulations! You have now conducted a catalog search for novae in M31.

Despite requiring detection in two images, your list of candidates should still be quite large. We will now work to prune this list to a more managable number so that we may inspect the individual candidates.

Problem 2) Reduce the Novae Candidate List

Our list of candidates still contains many false positives. There are some cuts we can apply to the list to make it more managable.

  • Cut 1 -- remove candidates that are fainter than \(R = 20 \textrm{mag}\).
    • magnitude measurements are stored in \(\texttt{SExtractor}\) catalogs as \(\texttt{MAG_AUTO}\).
  • Cut 2 -- require a high signal-to-noise ratio detection (\(SNR > 20\)).
    • the SNR can be measured by comparing the flux \(\texttt{FLUX_AUTO}\) to the fluxerr \(\texttt{FLUXERR_AUTO}\).
  • Cut 3 -- remove sources that \(\texttt{SExtractor}\) has flagged as bad.
    • \(\texttt{SExtractor}\) flags are stored in \(\texttt{FLAGS}\).

Apply these cuts to your candidate list. How many candidates do you now have? Print out the RA and Dec of each of your candidates. Can you think of anything else in the \(\texttt{SExtractor}\) catalogs that may be useful for eliminating false positives?

Hint - if you previously wrote your code using only RA and Dec for the catalogs, adjust it so that you can index your candidates while preserving all of the columns in the \(\texttt{SExtractor}\) catalogs.

In []:
### prune the list of candidates by making cuts on 
### brightness, SNR, and flags


Problem 3) Visually examine the candidates

Great! You now have a list of candidates that is small enough that you can inspect those candidates by eye. We will do this using \(\texttt{ds9}\), a software package designed for examining astronomical images. One benefit of \(\texttt{ds9}\) is that you load "region files", which will highlight certain things on an image to aid inspection.

A region file has the following format: # Region file format: DS9 version 4.1 global color=red dashlist=8 3 width=3 select=1 fk5 circle(ra1,dec1,10") circle(ra2,dec2,10") ... where \(\texttt{(ra1, dec1), (ra2, dec2), etc.}\) provide the coordinates of \(r = 10 \textrm{arcsec}\) circles that will be drawn over an image that is loaded into \(\texttt{ds9}\).

Using your list of candidates created above, create a \(\texttt{ds9}\) region file called \(\texttt{M31nova_cands.reg}\).

In [2]:
### create an M31 region file


Now that you have a region file - open the image from epoch 1 or epoch 2, and select "load regions" to overlay your region file on your fits images.

Do each of the highlighted candidates on your image look reasonable to you? Can you identify the best candidate?

The location(s) of the actual novae on this image will be revealed at the end of this interactive session.

Bonus Problem) Search for novae in M81

Finish early? We have multiple bonus problems for you -- including a black belt hard problem at the very end.

The Andromeda galaxy (M31) is very large (\(\sim 3 \deg\) across) and close to the Milky way. As a result, M31 novae are relatively bright, which, in turn, makes it easier to identify them.

Apply the same methodology that you used above to the catalogs and images of M81, a galaxy that is also very near the Milky Way, but \(\sim 5\) further away than M31. Do you find any good novae candidates in this case?

Not For the Faint of Heart

As you will find above - it is much more difficult to find a nova candidate in M81 using a catalog based search. Nevertheless, there is a genuine nova detected on the images of M81. To detect it you will need to use image subtraction. (Note - we do not reccommend you attempt this problem unless you are willing to become a little frustrated in the process. Image subtraction is hard.

Download HOTPANTS, software that performs image subtraction, and use it to perform image subtraction on the images of M81 that have been provided. Note - you will need to perform image registration (physically align the images you wish to subtract) prior to running the subtraction. One (of the many) ways to do this is using WCSremap, which, like \(\texttt{HOTPANTS}\) was written by Andy Becker at the University of Washington.

In []: