Volume Number: 18 (2002)
Issue Number: 11
Column Tag: QuickTime Toolkit

She's Gotta Have It

Using Media Sample References and Data References

by Tim Monroe

Imagine that we've got a couple of dozen pictures from a digital camera and that we'd like to create a slide show movie from those pictures -- that is, a QuickTime movie that displays each picture, in a predetermined sequence, for a predetermined amount of time. The first thing we'd need to do, of course, is create a new movie file, track, and media (by calling CreateMovieFile, NewMovieTrack, and NewTrackMedia). Then we might proceed like this: open each picture file, draw the picture data into an offscreen graphics world, compress the data, and add the compressed data as a new media sample by calling AddMediaSample. Then we would finish up by calling InsertMediaIntoTrack and AddMovieResource. Voila, we've got our slide show movie.

This strategy involves copying the picture data from the individual picture files into the slide show movie file, and for some purposes that might be exactly what we want to do. But if we're going to keep the picture files around anyway or if we're not sure we want to keep the resulting movie, it might be better to have our QuickTime movie file just point to the data in the picture files instead of having it contain a copy of that data. We can do this by inserting into the movie a media sample reference (or, more briefly, a sample reference) to the picture file data.

In this article, we're going to work with media sample references. We'll begin by taking a look at how to create media sample references, by developing a simple droplet application that creates a slide show movie from a number of picture files, as described above. As we'll see, sample references go hand in hand with data references, which we discussed at some length in an earlier QuickTime Toolkit article ("Somewhere I'll Find You" in MacTech, October, 2000). So we'll take this opportunity to investigate a couple of somewhat more advanced techniques for using data references. In particular, we'll see how to "flatten" a movie that contains a movie track, so that the child movie data is contained within the parent movie file; we'll also see how to create a movie whose media data is contained entirely in memory and then save it into a file.

In a nutshell, a media sample reference is a reference to some existing media data. The idea is that once we've got some media data stored in some location (a file, an object addressed by a URL, a block of memory, and so forth), we can reuse that media data by simply referring to it. That is, we don't need to copy the data in order to get access to it.

It's worth noting that we've bumped into sample references several times previously in this series of articles. Early on, when we discussed movie importers and exporters (in "In and Out" in MacTech, May, 2000), we learned that QuickTime can import some types of files without having to make a copy of the file data. We say that these kinds of files are imported in place -- meaning that the associated movie importer constructs a movie that directly references the data in the file being imported. Now we can see that a movie importer does this by inserting media sample references into the new movie. Those references point to the data in the imported file. Importing in place, by using media sample references, allows the new movie to be created more quickly and uses less storage space (since the media data does not need to be copied).

We also ran into sample references while we were working with the QuickTime video effects architecture, when we wanted to add a video effect to part of a video track. (See "F/X 2" in MacTech, October 2001.) The standard way to do this is to create a video new track that is a copy of the appropriate segment of the original video track; then we create an effects track that uses the copied track segment as its input, as shown in Figure 1.

The new video track can contain a copy of the media data in the original video track or it can contain only references to that media data.

Let's begin by reviewing the code we used to create the new track that we used as the input to our effects track. Listing 1 shows the relevant section of code.

QTEffects_AddEffectToMovieSegment
mySrcTrack1 = NewMovieTrack(theMovie, myWidth, myHeight, 
            kNoVolume);
if (mySrcTrack1 == NULL)
   return(paramErr);
mySrcMedia1 = NewTrackMedia(mySrcTrack1, VideoMediaType, 
            myTimeScale, NULL, 0);
if (mySrcMedia1 == NULL)
   return(paramErr);
#if COPY_MOVIE_MEDIA
myErr = BeginMediaEdits(mySrcMedia1);
if (myErr != noErr)
   return(myErr);
#endif
myErr = CopyTrackSettings(myVidTrack1, mySrcTrack1);
myErr = InsertTrackSegment(myVidTrack1, mySrcTrack1, 
            theStartTime, theDuration, theStartTime);
if (myErr != noErr)
   return(myErr);
#if COPY_MOVIE_MEDIA
EndMediaEdits(mySrcMedia1);
#endif

We call InsertTrackSegment to copy part of the original video track (myVidTrack1) into the track that will be used as the source for the effect (mySrcTrack1). If the compiler flag COPY_MOVIE_MEDIA is set to 0, then we don't call BeginMediaEdits and EndMediaEdits to begin and end a media-editing session; in this case, the new track contains references to the media data in the original track, thereby minimizing the resulting file size.

So, one way to create sample references is to call InsertTrackSegment without having opened a media-editing session (that is, without having called BeginMediaEdits). A more direct way to create sample references is to use the functions AddMediaSampleReference or AddMediaSampleReferences. Both of these functions allow us to add to a media one or more sample references to some existing data (and hence the names are slightly misleading). The main difference between these two functions is that when are adding a large number of samples to a movie at one time, AddMediaSampleReferences is significantly more efficient than AddMediaSampleReference. In this article, we will be adding only one sample reference at a time, so we'll restrict our attention to AddMediaSampleReference, which is declared essentially like this:

OSErr AddMediaSampleReference (
            Media theMedia, 
            long dataOffset, 
            unsigned long size, 
            TimeValue durationPerSample, 
            SampleDescriptionHandle sampleDescriptionH, 
            long numberOfSamples, 
            short sampleFlags, 
            TimeValue *sampleTime);

The parameters here are identical to those for AddMediaSample, with one exception: AddMediaSample also takes a handle to the data that is to be added to the media. With AddMediaSampleReference, we're not adding any data, so we don't need that parameter.

Here's how we might call AddMediaSampleReference to add a single media sample reference to a media:

myErr = AddMediaSampleReference(myMedia, myDataOffset, 
            mySize, myDuration, myDesc, 1, 0, NULL);

As you can see, the numberOfSamples parameter is set to 1, the sampleFlags parameter is set to 0, and the sampleTime parameter is set to NULL (since we don't care to have the new sample time returned to us). The other parameters are set to specific values determined by the application. For example, myDataOffset should specify the offset into the referenced media file (or other storage device) of the desired media data. In our sample slide show making application, myDuration will always be set to 600, so that each slide is displayed for one second.

Now, how does AddMediaSampleReference know where the original media data resides? As you know, QuickTime uses data references as its principal means of identifying the location of some data. So we might have expected that AddMediaSampleReference would take a data reference as a parameter. But, alas, you can see above that there is no such parameter. Instead, we need to attach the data reference to the media before we call AddMediaSampleReference; we do this by calling the AddMediaDataRef function, like this:

myErr = AddMediaDataRef(myMedia, &myDataRefIndex, 
            (Handle)myAlias, rAliasType);

AddMediaDataRef adds the specified data reference to the specified media and returns the index of that data reference in the media's list of data references. Then we assign that index to the dataRefIndex field of the sample description that we pass to AddMediaSampleReference:

(**myDesc).dataRefIndex = myDataRefIndex;

In this way, we've given AddMediaSampleReference a complete specification of the location of the data for which it will create a sample reference.

QuickTime also provides the functions GetMediaSampleReference and GetMediaSampleReferences, which we can use to get information about one or more samples that are stored in a media data file (or other media storage container). Unlike GetMediaSample, GetMediaSampleReference does not return the actual media data to us; rather, it gives us a handful of pieces of information about the media sample(s), such as the offset within the media container of the sample data, the size of the sample data, and a sample description that specifies the format of that sample data. We saw just above that we'll need that offset and size when we call AddMediaSampleReference; we'll also need a sample description when we're building our slide show movie. So we'll call GetMediaSampleReference like this:

myErr = GetMediaSampleReference(myRefMedia, 
            &myDataOffset, &mySize, 0, NULL, NULL, myDesc, 
            NULL, 1, NULL, 0);

Here, myRefMedia is the media to which we have a reference. In our slide show example, it'll be the data in the individual picture files.

Let's illustrate how to work with media sample references by building a slide show movie from a collection of picture files. We want the user to be able to drop any number of picture files onto our application; when this happens, the application will ask the user to specify a file name and location for the slide show movie; then it will create the movie and exit. Let's call this droplet DropPix.

The first thing that DropPix needs to do is assemble a list of the files that the user has dropped onto its icon. In our Macintosh application, we can do this inside of our AppleEvent handler for the Open Document event, as shown in Listing 2.

QTApp_HandleOpenDocumentAppleEvent
// open each specified file
for (myIndex = 1; myIndex <= myItemsInList; myIndex++)
   if (myErr == noErr) {
      myErr = AEGetNthPtr(&myDocList, myIndex, typeFSS, 
            &myKeyWd, &myTypeCode, (Ptr)&myFSSpec, 
            sizeof(myFSSpec), &myActualSize);
      if (myErr == noErr) {
         gSpecs[myIndex - 1] = myFSSpec;
      }
   }
if (myDocList.dataHandle)
   myIgnoreErr = AEDisposeDesc(&myDocList);
gNumSpecs = myIndex - 1;
DropPix_MakeSlideShow();
QTFrame_QuitFramework();         // act like a droplet and close automatically

You'll notice that we're using two global variables, gSpecs and gNumSpecs, to keep track the file system specification records for the dropped files. We declare those variables like this:

FSSpec               gSpecs[kMaxNumPictureFiles];
short                gNumSpecs;

(I'll leave it as an exercise for the enterprising reader to get rid of the hard-coded array size.)

On Windows, we can get a list of the dropped files by reworking some of the code in the QTFrame_OpenCommandLineMovies function (defined in the file WinFramework.c). First we need to remove the existing call to SHGetFileInfo that restricts our application to opening only QuickTime movie files. Then, once we've finished creating a file system specification record for a dropped file, we can add it to our array and increment our count of dropped files, as shown in Listing 3.

QTFrame_OpenCommandLineMovies
// make an FSSpec record
NativePathNameToFSSpec(myFileName, &myFSSpec, 
            kFullNativePath);
gSpecs[myFileIndex] = myFSSpec;
myFileIndex++;

When we are done collecting the files, we finish up like this:

gNumSpecs = myFileIndex;
DropPix_MakeSlideShow();

The DropPix_MakeSlideShow function first elicits a movie file name and location from the user, so that DropPix knows where to put the output slide show movie. Then it creates the new movie file:

myErr = CreateMovieFile(&myFile, sigMoviePlayer, 
            smCurrentScript, myFlags, &myResRefNum, &myMovie);
And, as usual, we'll create a new track and media:
myTrack = NewMovieTrack(myMovie, myWidth, myHeight, 0);
myMedia = NewTrackMedia(myTrack, VideoMediaType, 600, NULL, 
            0);

The bulk of DropPix_MakeSlideShow is a for loop that adds to this new media a sample reference to the data in each of the picture files in gSpecs.

Before we can call AddMediaSampleReference, we need to get the offset of the picture data in its file, and we need to add a data reference for that file to the new media. Adding a data reference is easy; first we create a file data reference:

myErr = QTNewAlias((const FSSpec *)&gSpecs[myIndex], 
            &myAlias, true);

Then we add it to the media:

myErr = AddMediaDataRef(myMedia, &myDataRefIndex, 
            (Handle)myAlias, rAliasType);

How do we get the offset of the data in the picture file? Earlier, we saw that we could use GetMediaSampleReference to get information about samples that are stored in a media data file. So all we need to do is call NewMovieFromDataRef to open the picture file as a movie and then call GetMediaSampleReference on that movie's media. Listing 4 shows the sequence of calls here.

DropPix_MakeSlideShow
// allocate a sample description
myDesc = (SampleDescriptionHandle)NewHandle(0);
myErr = MemError();
if (myErr != noErr)
   goto bailLoop;
myErr = NewMovieFromDataRef(&myRefMovie, 
            newMovieDontResolveDataRefs, NULL, (Handle)myAlias, 
            rAliasType);
if (myErr != noErr)
   goto bailLoop;
// get the first track's media
myRefTrack = GetMovieIndTrack(myRefMovie, 1);
myRefMedia = GetTrackMedia(myRefTrack);
if ((myRefTrack == NULL) || (myRefMedia == NULL))
   goto bailLoop;
myErr = GetMediaSampleReference(myRefMedia, &myDataOffset, 
            &mySize, 0, NULL, NULL, myDesc, NULL, 1, NULL, 0);

This gives us the data offset and the media sample size. It also gives us a sample description of the image, which we can use (for instance) to determine the size of the video track in the slide show movie. Our call to NewMovieTrack really looks like this:

myTrack = NewMovieTrack(myMovie, 
         Long2Fix((**(ImageDescriptionHandle)myDesc).width), 
         Long2Fix((**(ImageDescriptionHandle)myDesc).height), 
         kNoVolume);

Now we're almost finished. We add a sample reference to the data in the picture file like this:

(**myDesc).dataRefIndex = myDataRefIndex;
myErr = AddMediaSampleReference(myMedia, myDataOffset, 
            mySize, myDuration, myDesc, 1, 0, NULL);

Once we've done this for each picture file dropped onto our application, we need to call InsertMediaIntoTrack and AddMovieResource in the usual way. Listing 5 shows DropPix_MakeSlideShow in its full glory.

DropPix_MakeSlideShow
OSErr DropPix_MakeSlideShow (void)
{
   Movie         myMovie = NULL;
   Track         myTrack = NULL;
   Media         myMedia = NULL;
   FSSpec         myFile;
   Boolean      myIsSelected = false;
   Boolean      myIsReplacing = false;
   StringPtr   myPrompt = 
            QTUtils_ConvertCToPascalString("Save movie as:");
   StringPtr   myFileName = 
            QTUtils_ConvertCToPascalString("Untitled.mov");
   long            myFlags = createMovieFileDeleteCurFile | 
                                 createMovieFileDontCreateResFile;
   short         myResRefNum = kInvalidFileRefNum;
   short         myResID = movieInDataForkResID;
   short         myIndex;
   OSErr         myErr = noErr;
   if (gNumSpecs <= 0)
      return(paramErr);
   // prompt the user for new file name
   QTFrame_PutFile(myPrompt, myFileName, &myFile, 
            &myIsSelected, &myIsReplacing);
   myErr = myIsSelected ? noErr : userCanceledErr;
   if (myErr != noErr)
      goto bail;
   // delete any existing file of that name
   if (myIsReplacing) {
      myErr = DeleteMovieFile(&myFile);
      if (myErr != noErr)
         goto bail;
   }
   // create a movie file for the destination movie
   myErr = CreateMovieFile(&myFile, sigMoviePlayer, 
            smCurrentScript, myFlags, &myResRefNum, &myMovie);
   if (myErr != noErr)
      goto bail;
   // add a sample reference for each image to the new movie
   for (myIndex = 0; myIndex < gNumSpecs; myIndex++) {
      short            myDataRefIndex = 0;
      long               myDataOffset, mySize;
      SampleDescriptionHandle
                        myDesc = NULL;
      TimeValue      myDuration = 600;
      AliasHandle   myAlias = NULL;
      Movie            myRefMovie = NULL;
      Track            myRefTrack = NULL;
      Media            myRefMedia = NULL;
      myErr = QTNewAlias((const FSSpec *)&gSpecs[myIndex], 
            &myAlias, true);
      if (myErr != noErr)
         goto bailLoop;
      // allocate sample description
      myDesc = (SampleDescriptionHandle)NewHandle(0);
      myErr = MemError();
      if (myErr != noErr)
         goto bailLoop;
      myErr = NewMovieFromDataRef(&myRefMovie, 
            newMovieDontResolveDataRefs, NULL, (Handle)myAlias, 
            rAliasType);
      if (myErr != noErr)
         goto bailLoop;
      // get the first track's media
      myRefTrack = GetMovieIndTrack(myRefMovie, 1);
      myRefMedia = GetTrackMedia(myRefTrack);
      if ((myRefTrack == NULL) || (myRefMedia == NULL))
         goto bailLoop;
      myErr = GetMediaSampleReference(myRefMedia, 
            &myDataOffset, &mySize, 0, NULL, NULL, myDesc, 
            NULL, 1, NULL, 0);
      if (myErr != noErr)
         goto bailLoop;
      if (myTrack == NULL) {
         // create the movie track and media
         myTrack = NewMovieTrack(myMovie, 
         Long2Fix((**(ImageDescriptionHandle)myDesc).width), 
         Long2Fix((**(ImageDescriptionHandle)myDesc).height), 
         kNoVolume);
         myErr = GetMoviesError();
         if (myErr != noErr)
            goto bail;
         myMedia = NewTrackMedia(myTrack, VideoMediaType, 600, 
            NULL, 0);
         myErr = GetMoviesError();
         if (myErr != noErr)
            goto bail;
      }
      // add a data reference to the media
      myErr = AddMediaDataRef(myMedia, &myDataRefIndex, 
            (Handle)myAlias, rAliasType);
      if (myErr != noErr)
         goto bailLoop;
      (**myDesc).dataRefIndex = myDataRefIndex;
      // add a media sample reference to the media
      myErr = AddMediaSampleReference(myMedia, myDataOffset, 
            mySize, myDuration, myDesc, 1, 0, NULL);
bailLoop:
      if (myDesc)
         DisposeHandle((Handle)myDesc);
   }
   // add the media to the track
   myErr = InsertMediaIntoTrack(myTrack, 0, 0, 
            GetMediaDuration(myMedia), fixed1);
   if (myErr != noErr)
      goto bail;
   // add the movie atom to the movie file
   myErr = AddMovieResource(myMovie, myResRefNum, &myResID, 
            NULL);
bail:
   if (myResRefNum != kInvalidFileRefNum)
      CloseMovieFile(myResRefNum);
   if (myMovie != NULL)
      DisposeMovie(myMovie);
   free(myPrompt);
   free(myFileName);
   return(myErr);
}

As you can see, the size of the slide show video track is determined by the size of the first picture file in gSpecs. Each remaining image is scaled to fit into that track rectangle, which may result in some distortion of the image. I'll leave it as an exercise for the reader to figure out a way to avoid that distortion.

A movie track is a track of type MovieMediaType that effectively embeds one movie inside of another, as illustrated in Figure 2. The key feature of using movie tracks -- instead of just layering one track on top of another track -- is that the parent and child movies can have different time bases, so they can (for instance) have different playback rates and different looping characteristics.

You may recall from our earlier discussion of movie tracks (in "Atomic Cafe" in MacTech, September 2000) that a media sample in a movie track consists of an atom container whose atoms specify the movie that is to be embedded in the main movie, as well as some of the playback characteristics of the embedded movie. The media sample does not typically contain the data for the child movie itself; rather, it points to that data using an atom of type kMovieMediaDataReference; this atom contains a data reference to the child movie data, which is most often a file data reference or a URL data reference.

It would be nice if there were a way to save a movie that contains a movie track, so that all the media data -- for the parent and the child movies -- is contained within a single file. This is actually quite straightforward, using a data reference extension. The essential idea is to simply append all of the child movie data to the data reference atom in the appropriate media sample. By suitably reconfiguring the data reference atom, we can force QuickTime to look in the data reference extension for the child movie data instead of resolving the data reference in that atom.

In the process of reconfiguring the data reference atom, we'll need to use a few techniques that are interesting in their own right. We'll need to learn how to get the data for a media sample at a specific movie time, and we'll need to learn how to replace an entire media sample in an existing movie. So let's get started.

A movie track (that is, a track of type MovieMediaType) can have one or more media samples, each of which is an atom container whose atoms pick out a child movie and specify its spatial layout and playback characteristics. We can get the media sample for a given movie time by calling GetMediaSample. The only "gotcha" is that GetMediaSample requires that this time be expressed in the media's time scale. To convert a movie time to the corresponding media time, we can call TrackTimeToMediaTime, as illustrated in Listing 6.

QTMIM_FlattenChildIntoParent
Handle                              mySample = NULL;
SampleDescriptionHandle      myDesc = NULL;
TimeValue                        myMovieTimeNow = 0;
TimeValue                        myMediaTimeNow = 0;
TimeValue                        myDuration = 0;
mySample = NewHandleClear(0);
if (mySample == NULL)
   return(MemError());
myDesc = (SampleDescriptionHandle)NewHandleClear(0);
if (myDesc == NULL) {
   myErr = MemError();
   goto bail;
}
myMovieTimeNow = GetMovieTime(myMovie, NULL);
if (myMovieTimeNow == GetMovieDuration(myMovie))
   myMovieTimeNow--;
myMediaTimeNow = TrackTimeToMediaTime(myMovieTimeNow, 
            myTrack);
if (myMediaTimeNow == -1) {
   myErr = invalidTime;
   goto bail;
}
myErr = GetMediaSample(myMedia, mySample, 0, NULL, 
            myMediaTimeNow, NULL, &myDuration, myDesc, NULL, 1, 
            NULL, NULL);

TrackTimeToMediaTime returns -1 if there is no media sample in the track at the specified movie time or if the specified movie time is outside the movie's active segment. Since we call GetMovieTime to get the current movie time, we're guaranteed that myMovieTimeNow will be within the active movie segment. (Notice that we decrement the movie time if we happen to be at the end of the movie.)

The time passed to GetMediaSample can be any time within the extent of the media sample. GetMediaSample retrieves the data for that sample and returns it in the handle we pass it (here, mySample). GetMediaSample also returns the duration of the media sample.

The media data, to repeat, is an atom container that holds at least a data reference atom that picks out the child movie. We can get that atom like this:

myAtom = QTFindChildByIndex(mySample, 
            kParentAtomIsContainer, kMovieMediaDataReference, 
            1, NULL);

If myAtom is non-zero, then we want to fetch the data in that atom. The data is a data reference prefixed by the data reference type. We can get the data reference and its type like this:

QTGetAtomDataPtr(mySample, myAtom, &myDataSize, 
            &myDataPtr);
myDataRefType = EndianU32_BtoN(*(OSType *)myDataPtr);
myErr = PtrToHand(myDataPtr + sizeof(OSType), 
            &myDataRef, myDataSize - sizeof(OSType));

If this is all successful, then myDataRefType is the data reference type and myDataRef is the data reference itself. Right now, we want to open the child movie specified by that data reference and load it completely into memory. To open the child movie, we can use NewMovieFromDataRef:

NewMovieFromDataRef(&myChildMovie, 0, NULL, myDataRef, 
            myDataRefType);

To load the movie's media data completely into memory, we can use FlattenMovieData, passing it a handle data reference. (We've used this trick previously, in "Somewhere I'll Find You", cited earlier.) Listing 7 shows the essential steps.

QTMIM_FlattenChildIntoParent
DataReferenceRecord         myDataRefRecord;
Handle                              myDataRefHandle = NULL;
Handle                              myHandleDataRef = NULL;
myDataRefHandle = NewHandleClear(0);
if (myDataRefHandle == NULL)
   goto bail;
myHandleDataRef = QTDR_MakeHandleDataRef(myDataRefHandle);
if (myHandleDataRef == NULL)
   goto bail;
myDataRefRecord.dataRefType = HandleDataHandlerSubType;
myDataRefRecord.dataRef = myHandleDataRef;
myMemoryMovie = FlattenMovieData(myChildMovie,
                              flattenFSSpecPtrIsDataRefRecordPtr | 
                                    flattenAddMovieToDataFork,
                              (FSSpecPtr)&myDataRefRecord,
                              sigMoviePlayer,
                              smSystemScript,
                              0L);
myErr = GetMoviesError();
if (myErr != noErr)
   goto bail;
DisposeMovie(myChildMovie);

Notice that the second parameter passed to FlattenMovieData contains the flattenFSSpecPtrIsDataRefRecordPtr flag, which indicates that the third parameter is a pointer to a data reference record, not a pointer to a file system specification record; it also contains the flattenAddMovieToDataFork flag, which tells FlattenMovieData to write the movie atom as well as the media data into the specified location. If we didn't specify flattenAddMovieToDataFork, we'd get only the media data in the child movie.

Recall that we want to replace the original data reference in the data reference atom in the child movie media sample by a new data reference that has the child movie data appended to it. That is to say, we want to attach a data reference extension to that original data reference. In this case, the extension is of type kDataRefExtensionInitializationData. Let's call this kind of extension an initialization data data reference extension -- or, more briefly, an initialization extension.

QuickTime uses an initialization extension in one case only: when the data reference is a handle data reference and the specified handle is NULL. When this happens, QuickTime takes the data directly from the initialization extension. In effect, we can use this type of data reference extension to short-circuit the normal data reference resolution that QuickTime would otherwise perform; we're saying: here is the data you're looking for, in this data reference extension.

So our task boils down to this: replace the existing data reference in the atom of type kMovieMediaDataReference by a handle data reference whose associated data is NULL; then append a data reference extension of type kDataRefExtensionInitializationData to that data reference. For this latter task, we'll use the utility function QTDR_AddInitDataDataRefExtension, defined in Listing 8.

QTDR_AddInitDataDataRefExtension
OSErr QTDR_AddInitDataDataRefExtension 
            (Handle theDataRef, Ptr theInitDataPtr)
{
   unsigned long      myAtomHeader[2];
   OSErr                  myErr = noErr;
   if (theInitDataPtr == NULL)
      return(paramErr);
   myAtomHeader[0] = EndianU32_NtoB(sizeof(myAtomHeader) + 
            GetPtrSize(theInitDataPtr));
   myAtomHeader[1] = EndianU32_NtoB(
            kDataRefExtensionInitializationData);
   myErr = PtrAndHand(myAtomHeader, theDataRef, 
            sizeof(myAtomHeader));
   if (myErr == noErr)
      myErr = PtrAndHand(theInitDataPtr, theDataRef, 
            GetPtrSize(theInitDataPtr));
   return(myErr);
}

Listing 9 shows the code we use to replace the original data reference atom by a new data reference atom that contains the data of the child movie.

QTMIM_FlattenChildIntoParent
// out with the old...
QTRemoveAtom(mySample, myAtom);
// ...and in with the new
myChildDataHandle = NewHandleClear(sizeof(OSType) + 
            sizeof(Handle));
if (myChildDataHandle != NULL) {
   OSType      myType;
   // set the data reference type
   myType = EndianU32_NtoB(HandleDataHandlerSubType);
   BlockMove(&myType, *myChildDataHandle, sizeof(OSType));
   // leave the next four bytes set to 0x00000000;
   // add a filenaming extension and an initialization extension
   myErr = QTDR_AddFilenamingExtension(myChildDataHandle, 
            NULL);
   if (myErr != noErr)
      goto bail;
   HLock(myDataRefHandle);
   myErr = QTDR_AddInitDataDataRefExtension 
            (myChildDataHandle, *myDataRefHandle);
   HUnlock(myDataRefHandle);
   if (myErr != noErr)
      goto bail;
   HLock(myChildDataHandle);
   myErr = QTInsertChild(mySample, kParentAtomIsContainer, 
            kMovieMediaDataReference, 1, 1, 
            GetHandleSize(myChildDataHandle), 
            *myChildDataHandle, NULL);
   HUnlock(myChildDataHandle);
   if (myErr != noErr)
      goto bail;
}

Notice that we need to add an empty filenaming extension before we add the initialization extension.

One final step remains, namely to replace the original media sample by the revised media sample. To do this, we first need to delete the track segment corresponding to the original media sample. Then we'll add the new media sample into the media and insert it into the track at that time.

It's easy enough to figure out the start time and duration of a particular media sample. Recall that we already have the current movie time stored in the variable myMovieTimeNow. We can then call GetTrackNextInterestingTime twice, asking it to search backward and forward for the boundaries of the current media sample:

GetTrackNextInterestingTime(myTrack, 
            nextTimeMediaSample | nextTimeEdgeOK, 
            myMovieTimeNow, -0x01000, &myMovieStartTime, NULL);
GetTrackNextInterestingTime(myTrack, 
            nextTimeMediaSample | nextTimeEdgeOK, 
            myMovieStartTime, 0x01000, NULL, &myMovieDuration);

Then we can call DeleteTrackSegment to delete the track segment occupied by the media sample:

DeleteTrackSegment(myTrack, myMovieStartTime, 
            myMovieDuration);

Then we proceed as usual, opening a media-editing session and adding the new media sample to the media. The key step here is a call to AddMediaSample:

myErr = AddMediaSample(myMedia, 
                  mySample,
                  0,                                 // no offset in data
                  GetHandleSize(mySample), 
                  myDuration,                  // frame duration
                  (SampleDescriptionHandle)myDesc, 
                  1,                                 // one sample
                  0,                                 // self-contained samples
                  &myNewTime);

Finally, we insert the media into the track at the desired time and for the desired duration:

myErr = InsertMediaIntoTrack(myTrack, myMovieStartTime, 
            myNewTime, myDuration, (Fixed)0x00010000L);

And we are done! Listing 10 shows the complete process in one handy routine.

QTMIM_FlattenChildIntoParent
OSErr QTMIM_FlattenChildIntoParent 
            (WindowObject theWindowObject)
{
   Movie                              myMovie = NULL;
   Track                              myTrack = NULL;
   Media                              myMedia = NULL;
   Handle                              mySample = NULL;
   SampleDescriptionHandle      myDesc = NULL;
   TimeValue                        myMovieTimeNow = 0;
   TimeValue                        myMediaTimeNow = 0;
   TimeValue                        myDuration = 0;
   QTAtom                              myAtom = 0;
   Ptr                                 myDataPtr = NULL;
   Handle                              myDataRef = NULL;
   long                                 myDataSize = 0;
   OSType                              myDataRefType;
   Movie                              myChildMovie = NULL;
   Movie                              myMemoryMovie = NULL;
   DataReferenceRecord            myDataRefRecord;
   Handle                              myDataRefHandle = NULL;
   Handle                              myHandleDataRef = NULL;
   Handle                              myChildDataHandle = NULL;
   Fixed                              myRate;
   TimeValue                        myMovieStartTime = 0;
   TimeValue                        myMovieDuration = 0;
   TimeValue                        myNewTime = 0;
   OSErr                              myErr = noErr;
   if (theWindowObject == NULL)
      return(paramErr);
   // round up the usual suspects: the parent movie, movie track, and movie track media
   myMovie = (**theWindowObject).fMovie;
   if (myMovie == NULL)
      return(invalidMovie);
   myTrack = GetMovieIndTrackType(myMovie, 1, 
            MovieMediaType, movieTrackMediaType);
   if (myTrack == NULL)
      return(invalidTrack);
   myMedia = GetTrackMedia(myTrack);
   if (myMedia == NULL)
      return(invalidMedia);
   // get the child movie sample data
   // if the parent movie is playing, stop it
   myRate = GetMovieRate(myMovie);
   SetMovieRate(myMovie, 0);
   mySample = NewHandleClear(0);
   if (mySample == NULL)
      return(MemError());
   myDesc = (SampleDescriptionHandle)NewHandleClear(0);
   if (myDesc == NULL) {
      myErr = MemError();
      goto bail;
   }
   myMovieTimeNow = GetMovieTime(myMovie, NULL);
   if (myMovieTimeNow == GetMovieDuration(myMovie))
      myMovieTimeNow--;
   myMediaTimeNow = TrackTimeToMediaTime(myMovieTimeNow, 
            myTrack);
   if (myMediaTimeNow == -1) {
      myErr = invalidTime;
      goto bail;
   }
   myErr = GetMediaSample(myMedia, mySample, 0, NULL, 
            myMediaTimeNow, NULL, &myDuration, myDesc, NULL, 1, 
            NULL, NULL);
   if (myErr != noErr)
      goto bail;
   // the media sample is an atom container; 
   // find the data reference atom inside the media sample
   myAtom = QTFindChildByIndex(mySample, 
            kParentAtomIsContainer, kMovieMediaDataReference, 
            1, NULL);
   if (myAtom != 0) {
      // get the data reference atom data
      QTLockContainer(mySample);
      myErr = QTGetAtomDataPtr(mySample, myAtom, &myDataSize, 
            &myDataPtr);
      if (myErr != noErr)
         goto bail;
      myDataRefType = EndianU32_BtoN(*(OSType *)myDataPtr);
      myErr = PtrToHand(myDataPtr + sizeof(OSType), 
            &myDataRef, myDataSize - sizeof(OSType));
      if (myErr != noErr)
         goto bail;
      QTUnlockContainer(mySample);
      // open the child movie and flatten it entirely into memory
      myErr = NewMovieFromDataRef(&myChildMovie, 0, NULL, 
            myDataRef, myDataRefType);
      if (myErr != noErr)
         goto bail;
      myDataRefHandle = NewHandleClear(0);
      if (myDataRefHandle == NULL)
         goto bail;
      myHandleDataRef = 
            QTDR_MakeHandleDataRef(myDataRefHandle);
      if (myHandleDataRef == NULL)
         goto bail;
      myDataRefRecord.dataRefType = HandleDataHandlerSubType;
      myDataRefRecord.dataRef = myHandleDataRef;
      myMemoryMovie = FlattenMovieData(myChildMovie,
                              flattenFSSpecPtrIsDataRefRecordPtr | 
                                    flattenAddMovieToDataFork,
                              (FSSpecPtr)&myDataRefRecord,
                              sigMoviePlayer,
                              smSystemScript,
                              0L);
      myErr = GetMoviesError();
      if (myErr != noErr)
         goto bail;
      DisposeMovie(myChildMovie);
      // replace the existing data reference atom by a "flattened" data reference atom
      // out with the old...
      QTRemoveAtom(mySample, myAtom);
      // ...and in with the new
      myChildDataHandle = NewHandleClear(sizeof(OSType) + 
            sizeof(Handle));
      if (myChildDataHandle != NULL) {
         OSType      myType;
         // set the data reference type
         myType = EndianU32_NtoB(HandleDataHandlerSubType);
         BlockMove(&myType, *myChildDataHandle, 
            sizeof(OSType));
         // leave the next four bytes set to 0x00000000;
         // add a filenaming extension and an initialization extension
         myErr = QTDR_AddFilenamingExtension
            (myChildDataHandle, NULL);
         if (myErr != noErr)
            goto bail;
         HLock(myDataRefHandle);
         myErr = QTDR_AddInitDataDataRefExtension
            (myChildDataHandle, *myDataRefHandle);
         HUnlock(myDataRefHandle);
         if (myErr != noErr)
            goto bail;
         HLock(myChildDataHandle);
         myErr = QTInsertChild(mySample, 
            kParentAtomIsContainer, kMovieMediaDataReference, 
            1, 1, GetHandleSize(myChildDataHandle), 
            *myChildDataHandle, NULL);
         HUnlock(myChildDataHandle);
         if (myErr != noErr)
            goto bail;
      }
      // add the new sample to the media
      // determine the bounds of this sample in movie time
      GetTrackNextInterestingTime(myTrack, 
            nextTimeMediaSample | nextTimeEdgeOK, 
            myMovieTimeNow, -0x01000, &myMovieStartTime, NULL);
      GetTrackNextInterestingTime(myTrack, 
            nextTimeMediaSample | nextTimeEdgeOK, 
            myMovieStartTime, 0x01000, NULL, &myMovieDuration);
      // splice this media over the old one
      DeleteTrackSegment(myTrack, myMovieStartTime, 
            myMovieDuration);
      myErr = BeginMediaEdits(myMedia);
      if (myErr != noErr)
         goto bail;
      // write a new media sample into the track
      myErr = AddMediaSample(myMedia, 
                        mySample,
                        0,                                 // no offset in data
                        GetHandleSize(mySample), 
                        myDuration,                  // frame duration
                        (SampleDescriptionHandle)myDesc, 
                        1,                                 // one sample
                        0,                                 // self-contained samples
                        &myNewTime);
      myErr = EndMediaEdits(myMedia);
      if (myErr != noErr)
         goto bail;
      // add the media to the track
      myErr = InsertMediaIntoTrack(myTrack, myMovieStartTime, 
            myNewTime, myDuration, (Fixed)0x00010000L);
   } else {
      myErr = cannotFindAtomErr;
   }
bail:
   if (mySample != NULL)
      DisposeHandle(mySample);
   if (myDesc != NULL)
      DisposeHandle((Handle)myDesc);
   if (myDataRef != NULL)
      DisposeHandle(myDataRef);
   if (myDataRefHandle != NULL)
      DisposeHandle(myDataRefHandle);
   if (myHandleDataRef != NULL)
      DisposeHandle(myHandleDataRef);
   if (myChildDataHandle != NULL)
      DisposeHandle(myChildDataHandle);
   // restore the original movie rate
   SetMovieRate(myMovie, myRate);
   return(myErr);
}

As written, QTMIM_FlattenChildIntoParent replaces the current movie media sample by a "flattened" sample. It would be easy to adapt this routine to iterate through all samples in the movie track and to flatten each child movie into the parent movie. I'll leave this refinement as an exercise for the reader.

Let's finish up this article by taking a look at a few ways we can create movies or tracks whose media data is contained entirely in memory. This is a useful thing to do for a number of reasons. For example, we might generate all of a movie's media data dynamically and not want to have to create a disk file to hold that data. Or we might want to add a track to an existing movie but don't want the track's media data to be added to the associated movie file unless the user explicitly requests it. In that case, we can tell the track to store its media data in memory, not in the original movie file.

The key element here is to create a handle data reference and to set it as the data reference for the movie (or track). Any media data written to the movie (or track) will be written into memory, at the location specified by the data reference.

In a previous article ("Somewhere I'll Find You", cited earlier), we saw how to create a movie whose associated media data is contained entirely in RAM. There, we took advantage of the fact that FlattenMovieData can flatten a movie into a location specified by a data reference instead of by a file system specification record. Listing 11 shows the core of our code for doing this. (This should remind you of Listing 7 above.)

QTApp_HandleMenu
Movie                           myNewMovie = NULL;
Handle                           myDataRef = NULL;
Handle                           myHandle = NULL;
DataReferenceRecord      myDataRefRecord;
myHandle = NewHandleClear(0);
if (myHandle == NULL)
   goto bail;
myDataRef = QTDR_MakeHandleDataRef(myHandle);
if (myDataRef == NULL)
   goto bail;
myDataRefRecord.dataRefType = HandleDataHandlerSubType;
myDataRefRecord.dataRef = myDataRef;
myNewMovie = FlattenMovieData(myMovie,
                           flattenFSSpecPtrIsDataRefRecordPtr,
                           (FSSpecPtr)&myDataRefRecord,
                           sigMoviePlayer,
                           smSystemScript,
                           0L);

Using FlattenMovieData assumes that we already have a movie (myMovie) and want to copy it entirely into RAM. It's sometimes also useful to create a new movie in RAM, using the NewMovie function and our standard calls to NewMovieTrack, NewTrackMedia, and so forth. The easiest way to do this is to set the movie's default data reference to a handle data reference, using the SetMovieDefaultDataRef function. We can call SetMovieDefaultDataRef like this:

myErr = SetMovieDefaultDataRef(myMovie, myDataRef, 
            HandleDataHandlerSubType);

Listing 12 defines a function, QTDR_CreateMovieInRAM, that creates a new movie whose media data is stored in a block of memory.

QTDR_CreateMovieInRAM
Movie QTDR_CreateMovieInRAM (void)
{
   Movie               myMovie = NULL;
   Track               myTrack = NULL;
   Media               myMedia = NULL;
   short               myResRefNum = 0;
   short               myResID = 0;
   Handle               myDataRef = NULL;
   Handle               myHandle = NULL;
   FSSpec               myFSSpec;
   OSErr               myErr = noErr;
   // create a new handle to hold the media data
   myHandle = NewHandleClear(0);
   if (myHandle == NULL)
      goto bail;
   // create a data reference to that handle
   myDataRef = QTDR_MakeHandleDataRef(myHandle);
   if (myDataRef == NULL)
      goto bail;
   myMovie = NewMovie(newMovieActive);
   if (myMovie == NULL)
      goto bail;
   myErr = SetMovieDefaultDataRef(myMovie, myDataRef, 
            HandleDataHandlerSubType);
   if (myErr != noErr)
      goto bail;
   // create the movie track and media
   myTrack = NewMovieTrack(myMovie, 
            FixRatio(kVideoTrackWidth, 1), 
            FixRatio(kVideoTrackHeight, 1), kNoVolume);
   myErr = GetMoviesError();
   if (myErr != noErr)
      goto bail;
   myMedia = NewTrackMedia(myTrack, VideoMediaType, 
            kVideoTimeScale, NULL, 0);
   myErr = GetMoviesError();
   if (myErr != noErr)
      goto bail;
   // create the media samples
   myErr = BeginMediaEdits(myMedia);
   if (myErr != noErr)
      goto bail;
   myErr = QTDR_AddVideoSamplesToMedia(myMedia, 
            kVideoTrackWidth, kVideoTrackHeight);
   if (myErr != noErr)
      goto bail;
   myErr = EndMediaEdits(myMedia);
   if (myErr != noErr)
      goto bail;
   // add the media to the track
   myErr = InsertMediaIntoTrack(myTrack, 0, 0, 
            GetMediaDuration(myMedia), fixed1);
   if (myErr != noErr)
      goto bail;
   // add the movie atom to the movie file
   AddMovieResource(myMovie, myResRefNum, &myResID, NULL);
bail:
   if (myDataRef != NULL)
      DisposeHandle(myDataRef);
   return(myMovie);
}

This function is virtually identical to other movie-creating functions we've seen in previous articles, except that it calls SetMovieDefaultDataRef to cause all media data to be written to a block of memory. Previously, we relied on the fact that a new movie's default data reference is the file opened by a call to CreateMovieFile or NewMovieFromFile. Here we are calling NewMovie to create a new movie with no attachment to any existing file, so we need to explicitly set the movie's default data reference.

If we want just a particular track to have its media data in memory, then we can pass a handle data reference when calling NewTrackMedia, like this:

myMedia = NewTrackMedia(myTrack, VideoMediaType, 
            myTimeScale, myDataRef, HandleDataHandlerSubType);

The specified data reference overrides the default movie data reference.

In all these cases, we've ended up with a movie that has some or all of its media data stored directly in memory, accessed using a handle data reference. We can, of course, play the movie, edit the movie, enable and disable tracks (and so forth), exactly as if the media data were contained in a file accessed via a file data reference or stored remotely and accessed via a URL data reference. But what happens if we want to save this movie into a file on disk? Well, it depends. If we created the movie entirely in memory (as in Listings 11 and 12), then our underlying application framework code will detect that no file is attached to the movie yet. In this case, it will elicit a filename from the user, create a new file (or delete and recreate an existing file), and then call FlattenMovieData to write the media data into the new file. FlattenMovieData strips out any unneeded media samples, resolves all remaining media sample references, and writes the media data into the movie file on disk. This movie file contains only file data references. It's a self-contained movie file that holds all of its media data. So far, so good.

Things start to get interesting, however, if we've already got a movie file attached to our movie. (This would happen, for instance, if we open an existing movie file and then add a track whose media data is accessed using a handle data reference.) In this case, when the user decides to save the movie, our application framework code calls UpdateMovieResource instead of FlattenMovieData. UpdateMovieResource does not write any media data into the movie file; rather, it simply updates the movie atom, which contains the data references for each media. The updated movie file now contains a handle data reference. The problem here is that, when the movie file is closed and then reopened, QuickTime won't be able to find the media data. The handle data reference, in all likelihood, no longer picks out any valid media data.

Certainly one way to avoid this problem is to make sure that we call FlattenMovieData at least once before we close a movie file. But that might not be desirable in some instances. For example, our movie might contain references to other files, in addition to the references to memory-based data. We might not want to force all data references to be resolved, just one or two of them.

As far as I know, QuickTime doesn't currently provide a way to flatten only selected tracks in a movie. We can work around this limitation, to some degree, by employing a simple technique involving initialization extensions. The idea is to "smuggle" a track's media data into the movie atom, by attaching that data to the media's data reference as an initialization extension. When we then call UpdateMovieResource, the media data will be written to the movie file, since it now forms part of the movie atom. The movie file once again contains a handle data reference, but it's harmless; when QuickTime reopens the movie file, it will notice the data reference extension and load the media data from that extension. Sweet.

It's actually quite easy to implement this smuggling. We do it by passing a handle data reference to NewTrackMedia, just as we did at the end of the previous section. But this time, instead of passing a handle data reference for a handle to a 0-length block of data, we'll pass a handle data reference for a NULL handle, where the handle data reference has an initialization extension. Let's begin by creating a handle data reference:

myDataRef = NewHandleClear(sizeof(Handle) + sizeof(char));

Remember that a handle data reference is a handle to a handle. Here we've created a handle to a 5-byte block of memory, all of whose bytes are set to 0. This represents a NULL handle and a 0-length filenaming extension. At this point, we'll tack on the atom header for the initialization extension:

myAtomHeader[0] = EndianU32_NtoB(sizeof(myAtomHeader));
myAtomHeader[1] = EndianU32_NtoB(
            kDataRefExtensionInitializationData);
myErr = PtrAndHand(myAtomHeader, myDataRef, 
            sizeof(myAtomHeader));

We haven't actually added any initialization data to the data reference extension, only the 8-byte atom header. But that's all we need at this point. We're ready to call NewTrackMedia:

myMedia = NewTrackMedia(myTrack, VideoMediaType, 
            kVideoTimeScale, myDataRef, 
            HandleDataHandlerSubType);

When QuickTime sees the initialization extension atom header in the handle data reference, it knows to keep the media data in the data reference itself, rather than in the memory block addressed by the handle that forms the first four bytes of the data reference's referring data. The handle data handler is going to allocate whatever memory is needed to hold the data we add to our media, so we can dispose of our data reference (myDataRef) immediately if we like.

Now we can edit the media and track as usual, for instance by calling AddMediaSample and InsertMediaIntoTrack. When we subsequently call UpdateMovieResource, the handle data handler will write out a handle data reference with a fully-configured initialization extension.

Listing 13 shows most of this assembled into a single routine, QTDR_CreateTrackInRAM. It adds a new video track to a movie, with the track's media data stored in RAM. In addition, the media data will be written into the data reference as an initialization extension when the movie atom is updated.

QTDR_CreateTrackInRAM
OSErr QTDR_CreateTrackInRAM (Movie theMovie)
{
   Track                  myTrack = NULL;
   Media                  myMedia = NULL;
   Handle                  myDataRef = NULL;
   unsigned long         myAtomHeader[2];
   OSErr                  myErr = noErr;
   if (theMovie == NULL)
      return(paramErr);
   myDataRef = NewHandleClear(sizeof(Handle) + 
                                             sizeof(char));
   if (myDataRef == NULL)
      return(MemError());
   myAtomHeader[0] = EndianU32_NtoB(sizeof(myAtomHeader));
   myAtomHeader[1] = EndianU32_NtoB(
            kDataRefExtensionInitializationData);
   myErr = PtrAndHand(myAtomHeader, myDataRef, 
            sizeof(myAtomHeader));
   if (myErr != noErr)
      goto bail;
   // create the movie track and media
   myTrack = NewMovieTrack(theMovie, 
            FixRatio(kVideoTrackWidth, 1), 
            FixRatio(kVideoTrackHeight, 1), kNoVolume);
   myErr = GetMoviesError();
   if (myErr != noErr)
      goto bail;
   myMedia = NewTrackMedia(myTrack, VideoMediaType, 
            kVideoTimeScale, myDataRef, 
            HandleDataHandlerSubType);
   myErr = GetMoviesError();
   if (myErr != noErr)
      goto bail;
   // create the media samples
   myErr = BeginMediaEdits(myMedia);
   if (myErr != noErr)
      goto bail;
   myErr = QTDR_AddVideoSamplesToMedia(myMedia, 
            kVideoTrackWidth, kVideoTrackHeight);
   if (myErr != noErr)
      goto bail;
   myErr = EndMediaEdits(myMedia);
   if (myErr != noErr)
      goto bail;
   // add the media to the track
   myErr = InsertMediaIntoTrack(myTrack, 0, 0, 
            GetMediaDuration(myMedia), fixed1);
bail:
   if (myDataRef != NULL)
      DisposeHandle(myDataRef);
   return(myErr);
}

This is a neat technique, but it's got a few limitations that you should know about. First of all, it works only with QuickTime 4.0 and later. Under earlier versions of QuickTime, data reference extensions are simply ignored. Also, and more importantly, because the initialization extension is stored inside the movie atom, you should avoid creating very large extensions. The extensions will remain in RAM for significant periods of time, so it's good to keep them small.

Data. We've gotta have it, at least if we want to do anything very interesting in our QuickTime movies. In this article, we've taken a look at a couple of useful ways of managing a movie's media data. First, we saw how to construct a movie that picks out its media data using media sample references. These references can refer to existing data that lives outside the movie file (as in the case of our slide show movie) or that is already contained in the movie file (as in the case of our effects movie). A sample reference is simply a way to make use of some existing data without having to copy it into a movie file or between tracks.

We've also seen, however, that it's sometimes useful to be able to go in the reverse direction, by forcing a movie's media data to be packed into an existing movie file. (At the very least, this makes it much easier to move the movie file around, since we don't need to worry about moving any other files that the movie depends upon.) Our standard means of doing this is to call FlattenMovieData, but sometimes that either doesn't work at all (as in the case of child movies) or doesn't work selectively enough for our purposes (as in the case of a single memory-based track). To work around some of the limitations of FlattenMovieData, we can use initialization data data reference extensions to attach media data directly to a data reference.

Tim Monroe in a member of the QuickTime engineering team. You can contact him at monroe@apple.com. The views expressed here are not necessarily shared by his employer.