Chapter 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22
Appendix A | Appendix B | Appendix C | Glossary | Index | Legal Stuff | License


Chapter 2 - Multimedia Technologies

Choosing an Architecture and Codec | Codec Central | Multimedia Architectures | Codecs | How Video Codecs Work | Codecs in Action | Difference Between Video and File Compression

As far as a computer is concerned, the two most important aspects of working with digital video are:

1.Synchronizing, managing, and playing movies
2.Making the movies small enough to be stored and played properly
In order to effectively handle these tasks, two special technologies were developed: multimedia architectures and codecs.

Multimedia architectures address the first issue of handling and synchronizing digital video files. The architectures Media Cleaner Pro 2.0 supports are: QuickTime, RealMedia, and VDO. Architectures are often referred to as formats, but architectures are much more than simply file formats.

Codecs handle the second issue by making movies small enough to play properly and be stored in a realistic amount of space. Codecs are the actual compression and decompression algorithms the word codec is short for compressors/decompressors. Cinepak is a commonly used codec for CD-ROM projects.

For a more detailed discussion of architectures and codecs, please see Appendix B.

The difference between architectures and codecs can sometimes be confusing, especially because some codecs are available within several different architectures. For example, there is a both a QuickTime and a Video for Windows version of Cinepak.

While the core compression algorithms may be the same with some cross-architecture codecs, the files made by different architectures are not usually interchangeable. For example, a QuickTime movie made with Cinepak won't play under Video for Windows, even though Video for Windows has a Cinepak implementation. However, some architectures provide for translation and playback of other architectures' formats. For example, ActiveMovie 1.0 will play both Video for Windows and QuickTime files.



Go back to the top of the page



Choosing an Architecture and Codec

Selecting the technologies for your projects is a two step process.

First, figure out which architecture best meets your needs. See Appendix B for the pros and cons of each architecture. For example, if you were making a cross-platform CD-ROM, QuickTime would probably be your best choice, since it is the only architecture available for both Macs and Windows with the capability to deliver digital video at the rate that CD-ROMs require.

Second, select the best codec for your needs. Once you select an architecture, that choice will dictate what codecs are available. You may want to purchase some additional codecs that provide better quality than the free ones. For more details on the various codecs, drop by Codec Central on our web site.

Go back to the top of the page



Codec Central

Codec Central is a web site Terran maintains to help you stay current with the various codecs and architectures. Codec Central has sample compressed video, more codec information, and other useful resources. As a Media Cleaner owner, you may also be eligible for special discounts on software.

To go to Codec Central, choose it from Media Cleaner's Internet menu, or go to: http://www.CodecCentral.com

Go back to the top of the page



Multimedia Architectures

The basics of the various architectures are listed below. Please see Appendix B for more details.


CD-ROM Architectures

Many developers are familiar with the CD-ROM architectures, since they have been available for several years. Apple's QuickTime is the most widely used format due to its robustness and cross-platform capabilities.

Microsoft's Video for Windows is sometimes used for Windows-only projects, and will be replaced by ActiveMovie at some point in the future.


World-Wide Web Architectures

All of the WWW architectures are changing rapidly. Media Cleaner Pro 2.0 currently supports QuickTime, RealMedia and VDOLive.

QuickTime movies can be placed on a site so that users can watch them as they download. No special server is needed, and the basic QuickTime components are free. However, purchasing a commercial codec such as ClearVideo will give you higher quality movies than the free QuickTime codecs.

RealMedia files are similar to QuickTime, but often look and sound better at a given data rate. Users can view RealMedia files as they download. Creating RealMedia files requires the purchase of the RealMedia encoder, and an optional video server may be bought to increase performance.

VDOLive is a "true streaming" format that automatically adjusts the video in real-time to the user's connection, so there is no waiting. You must purchase a VDO server to put VDOLive movies on your site.

Please visit Codec Central for more information.

Go back to the top of the page



Sound Formats

Media Cleaner supports two different sound formats - AIFF (Audio Interchange File Format) and RealAudio. Generally, AIFF is most widely used for CD-ROM projects, whereas RealAudio files are currently a better choice for WWW audio.

Go back to the top of the page



DVD-ROM

DVD-ROM is a new compact disc format that holds from 4.7 Gigabytes (4,700 Mb) to 17 Gigabytes (17,000 Mb) depending on details, such as if the disc is double-sided, etc. DVD-ROM is the computer-oriented format of DVD, and can be thought of as simply a huge 8x CD-ROM.

Many people are confused by the fact that there is also a "DVD-Video" format, which will be used by some new players for prerecorded movies. DVD-Video uses MPEG-2 compression, and is intended to replace VHS for movie rentals. DVD-Video is not directly applicable to computers at this time.

Unlike DVD-Video, DVD-ROM does not require you to use MPEG for your movies. You can use the same formats you would on a CD-ROM, only with higher data rates if you like. Many developers are looking at using QuickTime with higher data rates, while others are considering MPEG for their DVD-ROM projects.

DVD-ROM is very new, and does not have any substantial installed base. It will be several years before there are enough players on the market to support a standard commercial release, especially given the current high cost of the players. Many of the developers making DVD-ROM products today are planning to "bundle" them with DVD-ROM drives. Once DVD-ROM has a large installed base, it will be a much better solution for desktop video than CD-ROM given its higher data rates and much larger size.

Go back to the top of the page



MPEG

Media Cleaner is not able to currently produce MPEG files. However, with QuickTime's software based MPEG decoder, Media Cleaner is able to recompress MPEG video into other formats.

MPEG is a standard originally designed for consumer products, such as digital satellite dishes and DVD-Video discs. MPEG encompasses both compression algorithms (essentially codecs) as well as file format. Thus, it crosses the line between an architecture and a codec. It is currently possible to use MPEG within QuickTime, but it is somewhat difficult as the tools are in an early state.

NOTE: Version 1.0 of the QuickTime MPEG extension doesn't provide for separate handling of the audio track. This means that if you have an MPEG source movie, you will not be able to recompress the soundtrack into "normal" QuickTime for distribution. Because of this, MPEG source movies are often difficult to work with.

MPEG is excellent for consumer hardware, but has some drawbacks on the computer desktop. It is very CPU intensive, and most computers require special hardware to view full screen MPEG video. Thus, the installed base of MPEG-capable computers is relatively low. Software-only MPEG playback is possible, but requires a very high-end machine for good results. As powerful processors become more widely installed, this will improve the installed base of MPEG-capable computers.

Generally speaking, most multimedia developers don't yet use MPEG for titles, except for specialty CD-ROMs. MPEG is sometimes used for WWW video, but it wasn't designed for very low data rates, so codecs like ClearVideo generally look better or take less space. There are also many configuration issues with MPEG, so it is not as widely used as the other architectures described above.

Go back to the top of the page



Codecs

Video, in its raw form, takes up huge amounts of space. For example, one second of uncompressed NTSC video takes about 27 Megabytes (Mb) of disk space. In order to make desktop video feasible, video compression algorithms were created. Compression is the process by which a large movie file is reduced in size by the removal of redundant audio and video data. For more dramatic size reduction, less important data may also be removed, resulting in image and/or sound degradation.

The codec is the algorithm that handles the compression of your video or audio, as well as the decompression when it is played. Each architecture has certain codecs available within it, and some codecs are implemented in multiple architectures.

Media Cleaner offers you the ability to use any compatible codec for any of the supported architectures (QuickTime, RealMedia, VDO).

Go back to the top of the page



How Video Codecs Work

Most codecs attempt to compress the data using spatial and/or temporal compression techniques to remove redundant data as described below. What follows is a very basic overview; the actual details of how video codecs do this are far beyond the scope of this manual.

Spatial Compression

One method of compressing movies is to remove the redundant data within any given image. For example, if you had a newscaster in front of a static background, the backdrop might have large areas of a solid color, with many identical pixels.

Instead of specifying each pixel and its color, a codec can generalize by specifying the coordinates of the area and the area's color; it doesn't have to note all the little details. This manner of reducing the size of an image is called spatial compression.

The less detail there is in the image, the better the codec is able to generalize the image and compress it. Removing fine details in preprocessing can improve the spatial compression of an image. Video noise often looks like fine detail to a codec, and should be removed to improve spatial compression. Creating video with simple backgrounds will also improve how well the final movie compresses.

Temporal Compression

Another way to make a frame smaller is to look for changes between consecutive frames and then store just the differences instead of the entire image. The original reference frame from which these differences are based is called a keyframe. Keyframes contain the entire image, and look just like a normal picture. The frames based on the changes between frames are called delta frames, or difference frames. They only contain information for the areas that are different from the last frame, and are usually smaller than the keyframes.

In our newscaster example, the first frame of the movie would be a keyframe, and would contain the entire image. After this initial keyframe, there would follow a series of delta frames. These delta frames would only show the differences between the previous frame and the current frame. Most of this delta data would relate to how the pixels in the newscaster's face were different from the last frame, and if he moved his hands, etc. The delta frame wouldn't bother with the static background since it always remains the same as it was in the previous frame. Every second or so a new keyframe would be used to correct for slight cumulative errors in the delta frames.

This kind of compression looks at data over a period of time and is therefore called temporal compression. Video content that changes very little from frame to frame is best suited for temporal compression. Whenever possible, you should use a tripod when filming video for desktop playback and attempt to reduce camera and subject movement. This will minimize the differences between frames and can improve the final compression.

NOTE: Some codecs, such as ClearVideo, are able to detect moderate camera motion and compensate for pans and zooms. Using a tripod, and maintaining smooth camera motion with these codecs may still improve the image quality as compared to hand-held camera work.

Go back to the top of the page



Codecs in Action

The actual process of analyzing each frame and creating a compressed version is what takes so long in video compression - for each frame, vast numbers of mathematical calculations are made to generate the final compressed frame. On a midrange Power Macintosh, it can often take a couple of seconds to compress a single frame.

The codec also controls the playback of the compressed video. It's no accident that the decompression routines are usually much faster than the compression routines this allows the frames to be decompressed fast enough to play in real-time. A frame that took a couple of seconds to compress might take less than 1/30th of a second to decompress.

Codecs that take a long time to compress but decompress quickly are known as "asymmetric." For example, the Cinepak codec is extremely asymmetric. This means that movies made with Cinepak take a long time to create, but decompress in real-time and play smoothly.

Codecs that are intended for "live" broadcasts and teleconferencing are "symmetric," meaning they both compress and decompress in real-time. However, because these codecs don't have as long to optimize each frame during compression, the results often don't look as good as movies made with asymmetric codecs.

Go back to the top of the page



The Specific Codecs

There are many different codecs available, and more on the way. We've seen previews of some very exciting technologies that should be available in mid-to-late 1997.

For a brief description of the most common codecs, please see Appendix B. To view the various codecs and get more information about them, select the "Codec Central" option in Media Cleaner's Internet menu, or go to:http://www.CodecCentral.com

Go back to the top of the page



Understanding the Difference Between Video Compression and File Compression

Video compression is significantly different from lossless file compression as implemented by such programs as StuffIt.

With file compression, files are losslessly compressed so they take up less space on the hard drive. This is also very useful for transfer over the Web. These files must be decompressed by the original compression program to turn them back into normal files prior to being used. For example, you can't launch a Stuffit archive, except to translate it back into a normal Macintosh file. Once it's uncompressed, you can then run the file. Since file-compressed files aren't usable until they have been decompressed, this compression is an intermediate format.

In contrast, compressed movies are the final distribution files. Compressed movies do not need to be decompressed by an application prior to being played, because the decompression is handled in real-time by the codecs. Also, video compression is almost never lossless; the compressed image doesn't have as much data as the original. Because of this data loss, you should never compress the same movie multiple times ­ each compression lowers the image quality.

NOTE: Some people try to make their movie files even smaller for storage by using a file compression program such as Stuffit. This usually saves very little or no space because the original movie doesn't have much redundant data in it ­ the redundant data was already removed when the movie was compressed with the selected codec. Since file compression programs only losslessly remove data, and there is nothing left for them to take out, the files stay the same size even after they have been "stuffed."

Go back to the top of the page