Data Archives

Once your data is in a usable format, you will want to store the data in a place where astronomers from all over the world can access it. Before data is made public, it may go through a "proprietary phase." This is a period of time during which only the team of scientists who requested the observation will have access to the data. The proprietary phase typically lasts a year (though this varies by satellite), after which the data is available to any interested scientist.

During both the proprietary and public phases, the data is stored in one central location, enabling scientists from around the globe to access the data at any time. Proprietary data is password-protected until it becomes public. Goddard's High Energy Astrophysics Science Archive Research Center (HEASARC) currently archives data from 20 satellites covering 30 years of observations.

Why create an archive?

The primary purpose for an archive is to make data easily available to as many interested people as possible. An astronomical archive serves much the same purpose as a family album or vacation scrapbook. During a family vacation, you likely take many pictures and collect brochures and postcards from the sites you visited. After the vacation you collect these pictures, postcards and brochures in a scrapbook. The scrapbooks make it easy for you to look back through the pictures and to share your photos and vacation memories with friends and family.

In an astronomical archive, the vacation pictures, postcards, and brochures are replaced by images, lightcurves, spectra, and raw data of lots of different celestial objects. An astronomer looking through the archive may be conducting an historical study, performing theoretical follow-up, conducting a survey, or assuring that the data has been looked at and is not duplicated. Let's look at each of these uses for an archive in a bit more detail.

Perhaps the most obvious reasons to look through pictures taken years ago are to remember good times spent with your friends, to see how your family has changed over the years, and to see if you still have that picture of your graduation party. In this sense, the scrapbooks serve as a historic record of your life and the lives of people around you.

If you were an astronomer, you'd use an archive to trace the history of an astronomical object that you've recently observed. Such an archive search would give you a picture of how the object has changed over time, allowing you to determine if the source has displayed new characteristics since the previous observations.

In your new observation, you might discover a new phenomena; however, only a detailed search of archived observations of the same area of the sky would reveal whether or not the phenomena was truly new.

You might also use your photo albums in a "theoretical follow-up" study. This use of an archive may not be as obvious as historical studies, but it is quite important for astronomy and other sciences. Imagine that while you were thinking about your last family reunion, it suddenly struck you that all of the men at the reunion from your father's side of the family had curly brown hair. You might hypothesize that all of the men from your father's family have brown curly hair. If you didn't have a record of past family reunions, you would have to wait until the next reunion to start to test your hypothesis. However, with a family album, you could immediately look back through years of family reunions to try to verify your hypothesis.

As an astronomer, if you develop a new theory, you can go to archived data to test that theory against observations. Prior to such easy access to historical data, a theorist had to either befriend observers and wait for new observations to test their theories, or eyeball data from a printed journal. Now, with the data covering years of observations, theorists have ready access to many observations, enabling them to test their theories quickly with more reliable data.

Suppose you wanted to collect photos of all of your cousins on your mother's side to try to determine what characteristics they inherited from your grandma or grandpa. Such a "survey" could easily be done by searching through your family albums and making a big sample of cousin pictures.

As an astronomer, you might conduct such a survey of astronomical objects. You would collect many observations made of a single type of object, like supernova remnants, by a single instrument and telescope. A survey like this is a lot easier to do if you have access to an archive of all the observations from that telescope. Even before all of the observations of supernova remnants became public, you could produce a good sample of supernova remnants.

Imagine that you are planning a second visit to Mount Rushmore. You know that you already have a slew of pictures from your first trip. Before you go on your trip, you might look back through your scrapbook to see what you have pictures of, and which pictures didn't turn out very well. On your trip, you will certainly take more pictures of the presidents' heads, but this time you might remember some of your pictures from the last trip, and try something new. Maybe you will take pictures at a different time of day to get different lighting effects, or maybe you'll make sure that you are standing in a different place when you take the pictures to exploit a different angle. Your search of your earlier pictures will prevent you from exactly duplicating the pictures you already have in your album. Such a use for your scrapbooks is called "assurance".

An astronomical archive provides the community assurance that all observations have been analyzed and that unjustified duplicated observations are not made. Since observing time on satellites is expensive, it is important that all observations are at least analyzed, even if the results are not noteworthy enough to publish. If all the data are at least looked at, this will prevent unwarranted duplicate observations from being performed.

The HEASARC archive

The archive for high-energy astronomical observations is held at Goddard's High Energy Astrophysics Science Archive Research Center (HEASARC). The goal of the HEASARC archive has always been to provide the most up-to-date data files from any particular mission. To achieve this goal, the entire archive is available through FTP and the Web. In 2002, the HEASARC archive held over 1200 gigabytes of data, but this represents files that are compressed and would occupy about 3 times as much space if they were expanded. The HEASARC archive grows 1 gigabyte of compressed data every day. And every month, astronomers download about 200 Gb of data from the HEASARC website - that's 200,000 floppy disks worth of data!

Next: Tell me about Data Analysis