|
Technote 1130Introducing PuppetTime:
|
CONTENTSDefining
PuppetTime |
This Technote describes how to add new media types to QuickTime, and uses the concept of digital actors as an example. 3D media is very exciting to use, but it is beyond the ability of nmost users to create. I definitely believe that 3D is the medium of the future, yet I am constantly frustrated by the learning curve associated with high-end 3D modeling and animation tools. For professionals, these tools are the cream of the crop; however, I want something a little more progressive. I want to manipulate 3D objects that know how to animate themselves, and can interact with each other. Drag a dog onto a stage, then tell it to wag its tail. Drag a cat onto a stage, then tell it to walk around. Put the two together, and tell the dog to chase the cat. In short, I don't want to make 3D objects; I want to use 3D objects to create other things. That's why I've created the PuppetTime architecture. |
Defining PuppetTime |
QTAtoms
, defines a new component interface called puppets, and includes both a derived media handler and a movie import component. The PuppetTime framework is designed with a philosophy similar to QuickTime: it contains a set of toolbox routines for manipulating the PuppetTime media data, as well as a number of extensible components and component interfaces. Much like Sprites and QuickTime Music Architecture in QuickTime, PuppetTime can be used by itself in your applications, and it can also be contained in QuickTime movies alongside other media types like music, text, sound, and video.
Technology MapFigure 1 shows the basic QuickTime architecture with the integrated PuppetTime media type and its related components.![]() Figure 1. PuppetTime inside QuickTime. |
EventsLet's begin our detailed discussion of PuppetTime with events. As mentioned above, a PuppetTime event is a
|
QTAtomContainer aContainer = nil; QTAtom anAtom = nil; long aLong = 0; // create the QTAtomContainer anError = QTNewAtomContainer(&aContainer); // add some name-data pairs to the root aLong = 7; anError = QTInsertChild( aContainer, // the container 0, // the atom, zero = root 'data', // the name 1, // the ID 0, // the index of name-ID pairs sizeof(aLong), // the size of the data &aLong, // the pointer to the data nil); // returns a ref to the new QTAtom aLong = 3; anError = QTInsertChild(aContainer, 0, 'data', 2, 0, sizeof(aLong), &aLong, nil); // create an empty atom anError = QTInsertChild(aContainer, 0, 'more', 1, 0, 0, nil, &anAtom); // add some atoms to it aLong = 2; anError = QTInsertChild(aContainer, anAtom, 'stff', 1, 0, sizeof(aLong), &aLong, nil); aLong = 14; anError = QTInsertChild(aContainer, anAtom, 'xtra', 1, 0, sizeof(aLong), &aLong, nil); // make sure to dispose of it when you're done anError = QTDisposeAtomContainer(aContainer); |
Thus, a PuppetTime event is a QTAtom
structure with a well-defined set of name-data hierarchy, shown in Figure 3 (QTAtom
IDs are not used by the PuppetTime toolbox routines, and will be omitted from the following figures).
Figure 3. The PuppetTime event structure
A PuppetTime event includes the following atoms:
| Target - this atom contains an ID of the target puppet for this event. Puppet IDs are assigned to puppets at runtime, so any puppet can be used, and the event always goes to the puppet with the given ID. |
| Time - this atom contains the time that this event should occur. This time value will be relative either to the start time of a movie or to the system clock. The puppet is responsible for queuing this event until the given time has arrived. A queuing method is provided to any puppet that wants it (explained below). |
| Messages - each event contains one or more messages. Each message contains a message class/code combination and zero or more parameters. In this way, a number of messages can be sent to a puppet with only one event. |
| Class - contains the message class being invoked (e.g., 'core' or 'musi'). |
| Code - contains the message code, (e.g., 'walk' or 'wave'). |
| Data - each message can contain a number of parameters, as defined by the creator of the event suite. The parameters can augment the resulting behavior and/or animation (e.g., speed of walk, or exaggeration of wave). |
There are constants defined in the header file
PuppetTimeEvents.h
for the event and message names used to
build a PuppetTime event, to ensure that all events have the
same structure.
As noted above, you can use QuickTime toolbox routines to
build QTAtoms
as PuppetTime events, as long as you structure the QTAtoms
in the basic format described. Because the format is so specific, there are a number of PuppetTime toolbox routines that make creating and parsing PuppetTime events easy. Look at the file PuppetTimeEvents.h
for a complete list of available APIs.
As an example, let's describe a class of events that represent music. Figure 4 shows a typical music event structure:
Figure 4. A typical music event
Listing 2 shows how to use the PuppetTime toolbox routines to build the event shown in Figure 4. Notice the PuppetTime toolbox streamlines the hierarchical nature of the event for you.
Listing 2
QTAtomContainer MakeAMusicEvent() { SInt32 instrument = 1; SInt32 eventTime = 15; UInt16 noteNumber; UInt16 noteVelocity; QTAtomContainer anEvent = nil; QTAtomContainer aMessage = nil; OSStatus anError = noErr; // create a new message aMessage = PTNewMessage(kPTInstrumentClass, kPTNoteEvent); // insert the parameters noteNumber = 60; noteVelocity = 0 anError = PTSetProperty(aMessage, kNoteNumber, sizeof(noteNumber), ¬eNumber); anError = PTSetProperty(aMessage, kNoteVelocity, sizeof(noteVelocity), ¬eVelocity); // create the event anEvent = PTNewEvent(instrument, eventTime, aMessage); // add the second message noteNumber = 62; noteVelocity = 90; anError = PTSetProperty(aMessage, kNoteNumber, sizeof(noteNumber), ¬eNumber); anError = PTSetProperty(aMessage, kNoteVelocity, sizeof(noteVelocity), ¬eVelocity); anError = PTSetNthMessage(anEvent, 0, aMessage); anError = PTReleaseMessage(aMessage); // do something with the event, like add it to a track anError = PTReleaseEvent(anEvent); return anEvent; }
To minimize the size of PuppetTime event streams, there are a number of optimizations that can be made when storing or transmitting events.
The first optimization is the concept of default values. When an event is defined by an author in a header file, certain parameters will be defined to have default values. When a developer creates an event, she can omit certain parameters to save space. When the recipient of an event goes to read those parameters and finds none, she can assume a default value. So for the music event example above, the default value for the velocity is zero, and that "note off" messages can contain one less parameter. The savings can add up quickly in a PuppetTime track that represents a song visually. Default value optimizations should be done by the creator of the events.
The second optimization is the concept of event
flattening. Oftentimes an event will contain only one
message, so the contents of the message atom (class, code,
and parameters) are moved into the event atom, and the
message atom is removed. The PuppetTime toolbox routine
PTOptimizeEventList
performs event flattening.
PuppetsNow we turn our attention to PuppetTime puppets. PuppetTime defines a puppet component interface in the file The Puppet Component Interface Listing 3 shows the component interface for puppet components:
The key routines are described below.
Base PuppetAs you can see, there are a number of routines in a puppet component, and every puppet should implement all of them. But most puppets need the same internal organizations, like a queue for events that aren't quite ready for processing. Moreover, processing events and pulling out messages is the same for most puppets. To make the process of creating a new puppet for PuppetTime easier, most developers can create a derived puppet component. A derived puppet uses the services of a base puppet component as a delegate to its own code. Similar in concept to a base media handler or a base image decompressor, the base puppet component implements the basics of a puppet, and leaves the specifics of the geometries and animations to the developer. To create a derived puppet component, a developer must implement the following puppet component routines: The
In In
In the routine
When the base puppet decides to pull an event from its
queue, it reads each of the messages inside it and sends them to
As you can see, a base puppet handles a lot of th details for you, allowing you to concentrate on implementing your puppet's visual appearance and animations. Note also the object-oriented nature (i.e. inheritance and overriding) of using a base puppet inside your puppet. The Component Manager was designed specifically for this kind of use. Camera PuppetAnother important puppet in PuppetTime is the camera puppet. This is a derived puppet which provides a view of the PuppetTime world to the user. Any puppet can have a camera view associated with it, although it's not required. The camera puppet is special in that it has no geometry associated with it, and simply provides a view. The file Creating your own puppetsThere are several puppets with sample code available in the SDK, which demonstrate the proper way to use the base puppet component. Use these example projects as the basis for your puppet development. Make sure that you edit the 'thng' resource, and choose mixed- or upper-case constants for the subtype and manufacturer fields. At this time, there are no flags defined for puppets, so zero them out for now. When implementing puppets, you'll have to decide what vocabularies to support. There are several sets of vocabularies already defined in PuppetTime, such as core and music. The base puppet handles a number of the core events for you. You're also free to create your own vocabularies, but you should use your manufacturer code as the message class; you'll also have to generate PuppetTime tracks using your vocabularies. Music PuppetsFor music puppets, the file The PuppetTime MIDI import component, discussed below, creates PuppetTime tracks using the music vocabulary. This allows a user to quickly generate PuppetTime content by simply importing MIDI files into QuickTime movies using applications like MoviePlayer. |
ConductorNext we examine the heart of PuppetTime, the conductor. It is the central object that binds the puppets to the drawing environment and to the incoming event stream. 3D EnvironmentWhen an instance of the conductor component is created, it in turn instantiates a number of QuickDraw 3D objects to set up a drawing environment, including a renderer, viewer, context, etc. Each puppet is responsible for its own geometries, yet this information needs to be communicated to the conductor at some point. This is done when the conductor is instructed to draw: each puppet gets a chance to submit its geometries (and other objects) to the QuickDraw 3D rendering loop maintained by the conductor. Event DispatchingBesides being responsible for the overall display, the conductor is also responsible for dispatching events from an incoming events stream to the puppet instances. There is a class of events specific to the conductor, like the 'cast' event. Events targeted to the conductor have a target ID of zero. A cast event has the structure shown in Figure 5: ![]() Figure 5. A cast event structure The cast message contains the following fields:
When the conductor receives cast events, it examines the contents of the event to determine which puppet to instantiate, then creates and stores a puppet instance in its internal array. Cast events are among the first events passed to a conductor. Without them, there would be no puppets visible and nothing to dispatch further events to. You can use the routine We'll talk about cast events as they pertain to QuickTime movies below. Current CameraThe conductor has the concept of a current camera, which
is itself a puppet. When a conductor first initializes, and after it has created the QuickDraw 3D drawing environment, it casts a camera puppet and assigns it a special ID Events can be targeted to the default camera by using an
ID of |
The first part of this article described how PuppetTime is structured around QTAtoms and puppet components.Now let's focus on how PuppetTime is integrated with QuickTime. The initial release of PuppetTime allows for basic creation and playback of QuickTime movies containing PuppetTime tracks. |
A PuppetTime track in a QuickTime movie has the type 'PTmh'. Each sample in a PuppetTime track is a QTAtomContainer structure containing a list of PuppetTime events. The events in this list represents at minimum 2-3 seconds worth of animation; the duration of each sample can be much larger. This ensures that disk access is minimized for better playback performance. |
In QuickTime, each track type has a corresponding media handler component which handles the playback of the track contents. Thus, a video track is managed by an instance of the video media handler, and a sound track is managed by an instance of the sound media handler. |
The PuppetTime media type is no different: a PuppetTime track is managed by the PuppetTime media handler, which is a derived media handler. When QuickTime opens a movie and finds a PuppetTime track, it searches for the corresponding PuppetTime media handler (by finding a component of type 'mhlr' and a subtype equal to the track type, in this case 'PTmh') and creates an instance. |
Being a derived media handler, many of the functions are handled by the base media handler component. The PuppetTime media handler does handle certain routines itself, and the most notable are MediaInitialize and MediaIdle . |
MediaInitialize
functionDuring movie initialization, QuickTime calls the media handler routine MediaInitialize . Here, the PuppetTime media handler creates an instance of the PuppetTime conductor and tells it about the visual dimensions of the track. |
Next, it looks for some cast data associated with the track. The cast data is stored separately so that the cast of the movie can be easily changed. For example, in a PuppetTime track that contains music events, the actual puppets used during playback can be changed by modifying the cast data. The resulting visual representation will be different while the underlying event stream remains valid. Use the routines PTGetCast and PTSetCast to get and set the cast data for the track. |
MediaIdle
functionIf the conductor is the heart of PuppetTime, then the MediaIdle function is the heartbeat of a PuppetTime track. This routine is responsible for reading in samples from the underlying media and passing them off to the conductor for dispatch. It also gives idle time to the conductor so that it can draw. |
Of course, playing a PuppetTime track is only useful if you have a PuppetTime track. Creating a PuppetTime track in a QuickTime movie is simple. As explained above, each sample is a list of events; in this way, the samples are spaced apart such that the disk isn't accessed too often. Listing 8 shows the sample description record, which is rather uncomplicated. Listing 9 demonstrates the concepts behind adding a PuppetTime media to a movie. |
Listing 8
typedef struct PTMHDescription { long size; // Total size of struct long type; // kPTMediaType long resvd1; short resvd2; short dataRefIndex; long version; } PTMHDescription, *PTMHDescriptionPtr, **PTMHDescriptionHandle;
Listing 9
Track myTrack; Media myMedia; QTAtomContainer anEventList; PTMHDescriptionHandle aDesc = nil; TimeValue sampleTime; myTrack = NewMovieTrack(theMovie, (long) myWidth << 16, (long) myHeight << 16, 0); // create the media for the track myMedia = NewTrackMedia(myTrack, // the track kPTMediaType, // the type of media aTimeScale, // time scale nil, // data ref (OSType) nil); // type of data ref anEventList = PTNewEventList(); anError = PTAddLocateEventToList(anEventList, 1, // target 0, // time 0, // x 0, // y 0); // z anError = PTAddNoteEventToList(anEventList, 1, // target 5, // time 60, // note 95, // velocity 0); // duration (0=forever) anError = PTAddNoteEventToList(anEventList, 1, // target 65, // time 60, // note 0, // velocity (0=off) 0); // duration (0=forever) aDesc = (PTMHDescriptionHandle) NewHandleClear(sizeof(PTMHDescription)); (**aDesc).size = sizeof(PTMHDescription); (**aDesc).type = kPTMediaType; (**aDesc).version = kPTMediaVersion; // Start editing session anError = BeginMediaEdits(theMedia); // add the data to the media anError = AddMediaSample(theMedia, (Handle) anEventList, // the sample 0L, // offset GetHandleSize((Handle) anEventList), 65, // duration of sample (SampleDescriptionHandle) aDesc, 1, // number of samples 0, // sample flags &sampleTime); // returned time // end editing session anError = EndMediaEdits(theMedia); // append to the track anError = InsertMediaIntoTrack(theTrack, // the track -1, // where to insert 0, // where in the media 65, // how much media to insert 1L << 16); // the media rate
Of course, there are routines in the PuppetTime toolbox that make creating PuppetTime tracks even easier, like PTAddPuppetTimeSample and PTSetEventListForTrack . |
Users should have an easy way to create PuppetTime tracks from abundant existing content. The initial release of PuppetTime focuses on music visualization, and includes a movie import component that converts MIDI files into PuppetTime tracks. |
QuickTime already includes a movie import component for MIDI files. The trick is to hook into the QuickTime import component in such a way that while it is creating a music track, PuppetTime gets a chance to create a PuppetTime track alongside it. |
This can be done by capturing the MIDI import component and replacing it with the PuppetTime import component. This is a step beyond just delegating to a component. Capturing means that the PuppetTime import component gets exclusive use of the MIDI import component, and takes the latter out of the Component Manger's current registry. |
In addition, PuppetTime wants to capture the MIDI import component at startup time, so that whenever QuickTime starts to import a MIDI file--regardless of which application is calling QuickTime--the PuppetTime MIDI import component gets to do its magic. |
Capturing a component at runtime takes a bit of finesse. First, the thng resource must be properly configured: the type and subtype of the PuppetTime import component must match the component being capturing (in this case 'eat ' and 'Midi'). Next, the cmpWantsRegisterMessage flag is set to true, which tells the Component Manager that the PuppetTime import component wants its Register routine called at startup. The rest of the component flags should be the same as the component being capturing. Finally, the PuppetTime movie import component is a PPC-native component, so the component HasMultiplePlatforms flag is set to true. This tells the component manager to find the component in the extended 'thng' structure.
|
Now that the component successfully captures the MIDI import component, it needs to override the MovieExchangeImportFile function. This routine calls throughto the captured and delegated MIDI import component, which proceeds to create the music track from the MIDI file. After that routine returns, the PuppetTime import component then re-reads the MIDI file and creates a PuppetTime track, converting MIDI data structures into PuppetTime events using the music vocabulary. The code could have just as easily read the newly created music track. Either way, without any extra effort on the user's part, a new movie is created that contains both a music track and a PuppetTime track. |
To make the user experience complete, the PuppetTime
import component also overrides the MovieExchangeDoUserDialog routine. In this case, however, it doesn't call through to the MIDI import component, but puts up its own Options dialog instead.
|
Sample CodeIn the PuppetTime Sample Code, I've included slightly altered versions of the PuppetTime media handler and the PuppetTime movie import component for your review. You'll note that I've changed all occurrences of the subtype and manufacturer fields to 'XXXX' and 'YYYY'. If you choose to use these samples for the basis of your own projects, please change the constants to something more suitable. Also, please don't re-use my constants for the various PuppetTime components, particularly 'PTmh' for my media handler and media type; this will allow me to continue developing PuppetTime without external complications. |
Future DirectionMuch like the QuickTime architecture, PuppetTime is designed with future growth squarely in mind. More QuickTime integrationWith the initial release of the PuppetTime engine, only two QuickTime-related components are included: a media handler and a movie import component. As PuppetTime continues to develop and mature, more QuickTime components will be added.
Cross-platformOf course, creating a new media type, especially for web and internet applications, isn't as compelling unless it works on both Macintosh and Windows platforms. Near-term future development will focus on bringing the core toolbox and component functionality to both platforms under QuickTime 3.0. Consumer ApplicationsOf course, the PuppetTime architecture exists so that developers can create applications that create and edit the PuppetTime media type. A couple of consumer-level applications that I'd like to see happen are the Puppet Builder and the Puppet Scene Maker.The Puppet Builder application would allow a user to create new puppets, giving them shapes and simple animations, and matching animations to events. The Puppet Scene Maker would allow a user to create a scene with dialog in 3D. For example, you could drag puppets from a cast window onto a stage window, then enter dialog in the script window. You might also drag actions from a vocabulary window onto the script window to add movement, nuances, etc. |
Inside
Macintosh: QuickTime Components, by Apple Computer, Inc.
(Addison-Wesley, 1993).
3D
Graphics Programming With QuickDraw 3D, by Apple Computer, Inc.
(Addison-Wesley, 1995).
![]() |