Danny Amor's Online Library

home *** CD-ROM | disk | FTP | other *** search

/ Danny Amor's Online Library / Danny Amor's Online Library - Volume 1.iso / html / faqs / faq / mpeg-faq / part1 < prev next >

Wrap

Text File | 1995-07-25 | 48.9 KB | 1,015 lines

Subject: MPEG-FAQ: multimedia compression [1/6] Newsgroups: comp.graphics,comp.graphics.animation,comp.compression,comp.multimedia,alt.binaries.multimedia,alt.binaries.pictures.utilities,alt.binaries.pictures,alt.binaries.pictures.d,alt.answers,comp.answers,news.answers From: phade@cs.tu-berlin.de (Frank Gadegast) Date: 22 Aug 1994 12:28:13 GMT Archive-name: mpeg-faq/part1 Last-modified: 1994/08/22 Version: v 3.2 94/08/22 Posting-Frequency: bimonthly BEGIN -------------------- CUT HERE --------------------- 1/6 ==================================================== THE MPEG-FAQ [Version 3.2 - 22. Aug 1994] ==================================================== PHADE SOFTWARE Leibnizstr. 30, 10625 Berlin, GERMANY Inh. Frank Gadegast Fon/Fax: +49 30 3128103 phade@cs.tu-berlin.de http://www.cs.tu-berlin.de/~phade =========================================================================== This is my summary about MPEG. It's the fourth publication of this file. Lots of information has been added (which has surely brought errors with it, see Murphy's Law). This fourth addition is different to the previous ones. First: Some old sections have been removed, because they are old or there was nothing changing. So for a starter you can read the last MPEG-FAQ's (Version 1.1, 2.0, or 3.0) Get them via ftp from host: ftp.cs.tu-berlin.de (130.149.17.7) or quepasa.cs.tu-berlin.de file: /pub/msdos/dos/graphics/mpegfa11.zip file: /pub/msdos/dos/graphics/mpegfa20.zip file: /pub/msdos/dos/graphics/mpegfa30.zip This new FAQ will be there soon too, as 'mpegfa31.zip'. My MPEG-related software and my DOS-ports of several programs can be found there too. Second: The people where more interested to get the complete archives. Therefore the TRAIL-PACK-Service is still running. I'm still collecting EVERY info, video, sound or program. Get the Trail-Pack ! Third: MPEG-audio is coming ! There is source-code ! There is a brand-new written audio-section in this FAQ written by Harald Popp and Morton Hjerde, thnx to both. And even MPEG-2 things are coming ! Fourth: MPEG-interleaving (audio and video together you know !) is about to be the next step. The pretty first things are there, incl. interleaved streams. Fifth: The INDEX-files are excluded in this release of the MPEG-FAQ. You can ftp it from the same place you got this file from. It should be usally called MPGIDX31.ZIP ! It includes the INDEX-file (first picture of all known MPEG-movies) and the AINDEX-file (about 2 seconds of know MPEG-AUDIO-streams in one MPEG-stream) and text-indexes for movies and audio-files. Sixth: The MPEG Trailpack CD-Rom is there ! Get it ! More than 430 MB of movies, songs, documentation and utilities ! Read below, about how to Order ! Seventh:The newest FAQ can always be loaded using Mosaic from: http://www.cs.tu-berlin.de/~phade And surely, there are more interesting things to find ;o) Eigths: AND I HEARBY PROCLAIM, that: MPEG is getting SO important, that in about 5 years, you go to your HiFi-Dealer and you ask him if the TV or Stereo is capable of MPEG to make your descision, not if it works with CD's or HDTV !!! You should read carefully through this FAQ this time, cause lots of new information is hidden in all the sections. F.e. news about Dos, Amiga-, Atari-, OS/2-, Windows-, Windows-NT, VMS-, NeXT, Unix- and Mac-Players and coders !!! Read about the future of MPEG ... This summary is devided in 12 parts: I | WHAT IS MPEG-VIDEO/AUDIO ? II | PROFESSIONAL SOFTWARE III | PUBLIC-DOMAIN SOFTWARE OR SHAREWARE IV | MPEG-RELATED HARDWARE V | MAILBOX-ACCESS VI | FTP-ACCESS (PD) VII | MAIL-ORDER VIII | RETRIEVED MAIL OR ARTICLES IX | ADDITIONAL INFORMATION X | WHERE TO FIND MORE INFOS XI | NEWS XII | QUESTIONS I add my comments in brackets [], lines (---- or ====) seperate the chapters. Please try and find out more information yourself. I had enough to do by getting and preparing this information. And only bother me with file- request if its not possible for you to get it somewhere else !!! If you want to contribute to this FAQ in any way, please email me (probably by replying to this posting). My email address is: phade@cs.tu-berlin.de Or send any additional information via fax or e-mail. The fax is only reachable between Mo.-Fr. from 10.00-13.00 and from 15.00-18.30 german time. Phade (Frank Gadegast) DISCLAIMER: I HAVE NOTHING TO DO WITH THE NAMED COMPANIES, NO BUSINESS, IT'S JUST MY PERSONAL INTERESTED. THESE COMPANIES ARE NAMED, BECAUSE THEY ARE THE FIRST, BRINGING REAL MULTIMEDIA TO THE WORLD. SURE I MAKE ADVERTS FOR THEM WITH THIS FAQ, BUT HOPE- FULLY YOU, AS A READER OF THIS FAQ, WILL FORCE THEM TO PRODUCE MORE AND BETTER PRODUCTS. =========================================================================== I.1 | WHAT IS MPEG-VIDEO/VIDEO =============================== ------------------------------------------------------------------------------- I.1 | MPEG-Video ----------------- From comp.compression Mon Oct 19 15:38:38 1992 Sender: news@chorus.chorus.fr Author: Mark Adler <madler@cco.caltech.edu> [71] Introduction to MPEG (long) What is MPEG? Does it have anything to do with JPEG? Then what's JBIG and MHEG? What has MPEG accomplished? So how does MPEG I work? What about the audio compression? So how much does it compress? What's phase II? When will all this be finished? How do I join MPEG? How do I get the documents, like the MPEG I draft? [ There is no newer version of this part so far. Whoever wants to update ] [ this description, should do the job and send it over. ] ------------------------------------------------------------------------------ Subject: [71] Introduction to MPEG (long) Written by Mark Adler <madler@cco.caltech.edu>. Q. What is MPEG? A. MPEG is a group of people that meet under ISO (the International Standards Organization) to generate standards for digital video (sequences of images in time) and audio compression. In particular, they define a compressed bit stream, which implicitly defines a decompressor. However, the compression algorithms are up to the individual manufacturers, and that is where proprietary advantage is obtained within the scope of a publicly available international standard. MPEG meets roughly four times a year for roughly a week each time. In between meetings, a great deal of work is done by the members, so it doesn't all happen at the meetings. The work is organized and planned at the meetings. Q. So what does MPEG stand for? A. Moving Pictures Experts Group. Q. Does it have anything to do with JPEG? A. Well, it sounds the same, and they are part of the same subcommittee of ISO along with JBIG and MHEG, and they usually meet at the same place at the same time. However, they are different sets of people with few or no common individual members, and they have different charters and requirements. JPEG is for still image compression. Q. Then what's JBIG and MHEG? A. Sorry I mentioned them. Ok, I'll simply say that JBIG is for binary image compression (like faxes), and MHEG is for multi-media data standards (like integrating stills, video, audio, text, etc.). For an introduction to JBIG, see question 74 below. Q. Ok, I'll stick to MPEG. What has MPEG accomplished? A. So far (as of January 1992), they have completed the "Committee Draft" of MPEG phase I, colloquially called MPEG I. It defines a bit stream for compressed video and audio optimized to fit into a bandwidth (data rate) of 1.5 Mbits/s. This rate is special because it is the data rate of (uncompressed) audio CD's and DAT's. The draft is in three parts, video, audio, and systems, where the last part gives the integration of the audio and video streams with the proper timestamping to allow synchronization of the two. They have also gotten well into MPEG phase II, whose task is to define a bitstream for video and audio coded at around 3 to 10 Mbits/s. Q. So how does MPEG I work? A. First off, it starts with a relatively low resolution video sequence (possibly decimated from the original) of about 352 by 240 frames by 30 frames/s (US--different numbers for Europe), but original high (CD) quality audio. The images are in color, but converted to YUV space, and the two chrominance channels (U and V) are decimated further to 176 by 120 pixels. It turns out that you can get away with a lot less resolution in those channels and not notice it, at least in "natural" (not computer generated) images. The basic scheme is to predict motion from frame to frame in the temporal direction, and then to use DCT's (discrete cosine transforms) to organize the redundancy in the spatial directions. The DCT's are done on 8x8 blocks, and the motion prediction is done in the luminance (Y) channel on 16x16 blocks. In other words, given the 16x16 block in the current frame that you are trying to code, you look for a close match to that block in a previous or future frame (there are backward prediction modes where later frames are sent first to allow interpolating between frames). The DCT coefficients (of either the actual data, or the difference between this block and the close match) are "quantized", which means that you divide them by some value to drop bits off the bottom end. Hopefully, many of the coefficients will then end up being zero. The quantization can change for every "macroblock" (a macroblock is 16x16 of Y and the corresponding 8x8's in both U and V). The results of all of this, which include the DCT coefficients, the motion vectors, and the quantization parameters (and other stuff) is Huffman coded using fixed tables. The DCT coefficients have a special Huffman table that is "two-dimensional" in that one code specifies a run-length of zeros and the non-zero value that ended the run. Also, the motion vectors and the DC DCT components are DPCM (subtracted from the last one) coded. Q. So is each frame predicted from the last frame? A. No. The scheme is a little more complicated than that. There are three types of coded frames. There are "I" or intra frames. They are simply a frame coded as a still image, not using any past history. You have to start somewhere. Then there are "P" or predicted frames. They are predicted from the most recently reconstructed I or P frame. (I'm describing this from the point of view of the decompressor.) Each macroblock in a P frame can either come with a vector and difference DCT coefficients for a close match in the last I or P, or it can just be "intra" coded (like in the I frames) if there was no good match. Lastly, there are "B" or bidirectional frames. They are predicted from the closest two I or P frames, one in the past and one in the future. You search for matching blocks in those frames, and try three different things to see which works best. (Now I have the point of view of the compressor, just to confuse you.) You try using the forward vector, the backward vector, and you try averaging the two blocks from the future and past frames, and subtracting that from the block being coded. If none of those work well, you can intra- code the block. The sequence of decoded frames usually goes like: IBBPBBPBBPBBIBBPBBPB... Where there are 12 frames from I to I (for US and Japan anyway.) This is based on a random access requirement that you need a starting point at least once every 0.4 seconds or so. The ratio of P's to B's is based on experience. Of course, for the decoder to work, you have to send that first P *before* the first two B's, so the compressed data stream ends up looking like: 0xx312645... where those are frame numbers. xx might be nothing (if this is the true starting point), or it might be the B's of frames -2 and -1 if we're in the middle of the stream somewhere. You have to decode the I, then decode the P, keep both of those in memory, and then decode the two B's. You probably display the I while you're decoding the P, and display the B's as you're decoding them, and then display the P as you're decoding the next P, and so on. Q. You've got to be kidding. A. No, really! Q. Hmm. Where did they get 352x240? A. That derives from the CCIR-601 digital television standard which is used by professional digital video equipment. It is (in the US) 720 by 243 by 60 fields (not frames) per second, where the fields are interlaced when displayed. (It is important to note though that fields are actually acquired and displayed a 60th of a second apart.) The chrominance channels are 360 by 243 by 60 fields a second, again interlaced. This degree of chrominance decimation (2:1 in the horizontal direction) is called 4:2:2. The source input format for MPEG I, called SIF, is CCIR-601 decimated by 2:1 in the horizontal direction, 2:1 in the time direction, and an additional 2:1 in the chrominance vertical direction. And some lines are cut off to make sure things divide by 8 or 16 where needed. Q. What if I'm in Europe? A. For 50 Hz display standards (PAL, SECAM) change the number of lines in a field from 243 or 240 to 288, and change the display rate to 50 fields/s or 25 frames/s. Similarly, change the 120 lines in the decimated chrominance channels to 144 lines. Since 288*50 is exactly equal to 240*60, the two formats have the same source data rate. Q. You didn't mention anything about the audio compression. A. Oh, right. Well, I don't know as much about the audio compression. Basically they use very carefully developed psychoacoustic models derived from experiments with the best obtainable listeners to pick out pieces of the sound that you can't hear. There are what are called "masking" effects where, for example, a large component at one frequency will prevent you from hearing lower energy parts at nearby frequencies, where the relative energy vs. frequency that is masked is described by some empirical curve. There are similar temporal masking effects, as well as some more complicated interactions where a temporal effect can unmask a frequency, and vice-versa. The sound is broken up into spectral chunks with a hybrid scheme that combines sine transforms with subband transforms, and the psychoacoustic model written in terms of those chunks. Whatever can be removed or reduced in precision is, and the remainder is sent. It's a little more complicated than that, since the bits have to be allocated across the bands. And, of course, what is sent is entropy coded. Q. So how much does it compress? A. As I mentioned before, audio CD data rates are about 1.5 Mbits/s. You can compress the same stereo program down to 256 Kbits/s with no loss in discernable quality. (So they say. For the most part it's true, but every once in a while a weird thing might happen that you'll notice. However the effect is very small, and it takes a listener trained to notice these particular types of effects.) That's about 6:1 compression. So, a CD MPEG I stream would have about 1.25 MBits/s left for video. The number I usually see though is 1.15 MBits/s (maybe you need the rest for the system data stream). You can then calculate the video compression ratio from the numbers here to be about 26:1. If you step back and think about that, it's little short of a miracle. Of course, it's lossy compression, but it can be pretty hard sometimes to see the loss, if you're comparing the SIF original to the SIF decompressed. There is, however, a very noticeable loss if you're coming from CCIR-601 and have to decimate to SIF, but that's another matter. I'm not counting that in the 26:1. The standard also provides for other bit rates ranging from 32Kbits/s for a single channel, up to 448 Kbits/s for stereo. Q. What's phase II? A. As I said, there is a considerable loss of quality in going from CCIR-601 to SIF resolution. For entertainment video, it's simply not acceptable. You want to use more bits and code all or almost all the CCIR-601 data. From subjective testing at the Japan meeting in November 1991, it seems that 4 MBits/s can give very good quality compared to the original CCIR-601 material. The objective of phase II is to define a bit stream optimized for these resolutions and bit rates. Q. Why not just scale up what you're doing with MPEG I? A. The main difficulty is the interlacing. The simplest way to extend MPEG I to interlaced material is to put the fields together into frames (720x486x30/s). This results in bad motion artifacts that stem from the fact that moving objects are in different places in the two fields, and so don't line up in the frames. Compressing and decompressing without taking that into account somehow tends to muddle the objects in the two different fields. The other thing you might try is to code the even and odd field streams separately. This avoids the motion artifacts, but as you might imagine, doesn't get very good compression since you are not using the redundancy between the even and odd fields where there is not much motion (which is typically most of image). Or you can code it as a single stream of fields. Or you can interpolate lines. Or, etc. etc. There are many things you can try, and the point of MPEG II is to figure out what works well. MPEG II is not limited to consider only derivations of MPEG I. There were several non-MPEG I-like schemes in the competition in November, and some aspects of those algorithms may or may not make it into the final standard for entertainment video compression. Q. So what works? A. Basically, derivations of MPEG I worked quite well, with one that used wavelet subband coding instead of DCT's that also worked very well. Also among the worked-very-well's was a scheme that did not use B frames at all, just I and P's. All of them, except maybe one, did some sort of adaptive frame/field coding, where a decision is made on a macroblock basis as to whether to code that one as one frame macroblock or as two field macroblocks. Some other aspects are how to code I-frames--some suggest predicting the even field from the odd field. Or you can predict evens from evens and odds or odds from evens and odds or any field from any other field, etc. Q. So what works? A. Ok, we're not really sure what works best yet. The next step is to define a "test model" to start from, that incorporates most of the salient features of the worked-very-well proposals in a simple way. Then experiments will be done on that test model, making a mod at a time, and seeing what makes it better and what makes it worse. Example experiments are, B's or no B's, DCT vs. wavelets, various field prediction modes, etc. The requirements, such as implementation cost, quality, random access, etc. will all feed into this process as well. Q. When will all this be finished? A. I don't know. I'd have to hope in about a year or less. Q. How do I join MPEG? A. You don't join MPEG. You have to participate in ISO as part of a national delegation. How you get to be part of the national delegation is up to each nation. I only know the U.S., where you have to attend the corresponding ANSI meetings to be able to attend the ISO meetings. Your company or institution has to be willing to sink some bucks into travel since, naturally, these meetings are held all over the world. (For example, Paris, Santa Clara, Kurihama Japan, Singapore, Haifa Israel, Rio de Janeiro, London, etc.) Q. Well, then how do I get the documents, like the MPEG I draft? A. MPEG is a draft ISO standard. It's exact name is ISO CD 11172. The draft consists of three parts: System, Video, and Audio. The System part (11172-1) deals with synchronization and multiplexing of audio-visual information, while the Video (11172-2) and Audio part (11172-3) address the video and the audio compression techniques respectively. You may order it from your national standards body (e.g. ANSI in the USA) or buy it from companies like OMNICOM phone +44 438 742424 FAX +44 438 740154 ------------------------------------------------------------------------------- From: billd@cray.com (Bill Davidson) Subject: MPEG standards documents. Date: 21 Apr 94 02:16:32 MET I just connected to the Document Center WAIS server at wais.service.com to find out what MPEG documents cost. This is what I found: Title Pages Price(US$) ------------------------------------------------------- ----- ---------- ISO/IEC-11172-1 - PART 1: SYSTEMS, INFORMATION 60 158.75 TECHNOLOGY - CODING OF MOVING PICTURES & ASSOCIATED AUDIO FOR ISO/IEC-11172-2 - PART 2: VIDEO, INFORMATION TECHNOLOGY 122 204.00 - CODING MOVING PICTURES & ASSOCIATED AUDIO FOR DIGI ISO/IEC-11172-3 - PART 3: AUDIO, INFORMATION TECHNOLOGY 157 214.25 - CODING OF MOVING PICTURES & ASSOCIATED AUDIO FOR D ISO/IEC-CD-11172 - INFORMATION TECHNOLOGY - CODING OF 0 207.00 OF MOVING PICTURES & ASSOCIATED AUDIO - FOR DIGITAL STORAGE Is this a mistake or are standards documents really rediculously priced? Since these would be for my own personal use, I have to pay for them out of my own personal pocket. Just one of these eats my book budget for quite a while. I realize that they have to make money but this has got to be about a 1000% markup over printing costs; even assuming low volumes. Bill Davidson ------------------------------------------------------------------------------- I.2 | MPEG-Audio ----------------- From: "Harald Popp" <POPP@iis.fhg.de> From: mortenh@oslonett.no Date: Fri, 25 Mar 1994 19:09:06 +0100 Subject: Merged Modified MPEG audio FAQ Q. What is MPEG? A. MPEG is an ISO committee that proposes standards for compression of Audio and Video. MPEG deals with 3 issues: Video, Audio, and System (the combination of the two into one stream). You can find more info on the MPEG committee in other parts of this document. Q. I've heard about MPEG Video. So this is the same compression applied to audio? A. Definitely no. The eye and the ear... even if they are only a few centimeters apart, works very differently... The ear has a much higher dynamic range and resolution. It can pick out more details but it is "slower" than the eye. The MPEG committee chose to recommend 3 compression methods and named them Audio Layer-1, Layer-2, and Layer-3. Q. What does it mean exactly? A. MPEG-1, IS 11172-3, describes the compression of audio signals using high performance perceptual coding schemes. It specifies a family of three audio coding schemes, simply called Layer-1,-2,-3, with increasing encoder complexity and performance (sound quality per bitrate). The three codecs are compatible in a hierarchical way, i.e. a Layer-N decoder is able to decode bitstream data encoded in Layer-N and all Layers below N (e.g., a Layer-3 decoder may accept Layer-1,-2 and -3, whereas a Layer-2 decoder may accept only Layer-1 and -2.) Q. So we have a family of three audio coding schemes. What does the MPEG standard define, exactly? A. For each Layer, the standard specifies the bitstream format and the decoder. It does *not* specify the encoder to allow for future improvements, but an informative chapter gives an example for an encoder for each Layer. Q. What have the three audio Layers in common? A. All Layers use the same basic structure. The coding scheme can be described as "perceptual noise shaping" or "perceptual subband / transform coding". The encoder analyzes the spectral components of the audio signal by calculating a filterbank or transform and applies a psychoacoustic model to estimate the just noticeable noise-level. In its quantization and coding stage, the encoder tries to allocate the available number of data bits in a way to meet both the bitrate and masking requirements. The decoder is much less complex. Its only task is to synthesize an audio signal out of the coded spectral components. All Layers use the same analysis filterbank (polyphase with 32 subbands). Layer-3 adds a MDCT transform to increase the frequency resolution. All Layers use the same "header information" in their bitstream, to support the hierarchical structure of the standard. All Layers use a bitstream structure that contains parts that are more sensitive to biterrors ("header", "bit allocation", "scalefactors", "side information") and parts that are less sensitive ("data of spectral components"). All Layers may use 32, 44.1 or 48 kHz sampling frequency. All Layers are allowed to work with similar bitrates: Layer-1: from 32 kbps to 448 kbps Layer-2: from 32 kbps to 384 kbps Layer-3: from 32 kbps to 320 kbps Q. What are the main differences between the three Layers, from a global view? A. From Layer-1 to Layer-3, complexity increases (mainly true for the encoder), overall codec delay increases, and performance increases (sound quality per bitrate). Q. Which Layer should I use for my application? A. Good Question. Of course, it depends on all your requirements. But as a first approach, you should consider the available bitrate of your application as the Layers have been designed to support certain areas of bitrates most efficiently, i.e. with a minimum drop of sound quality. Let us look a little closer at the strong domains of each Layer. Layer-1: Its ISO target bitrate is 192 kbps per audio channel. Layer-1 is a simplified version of Layer-2. It is most useful for bitrates around the "high" bitrates around or above 192 kbps. A version of Layer-1 is used as "PASC" with the DCC recorder. Layer-2: Its ISO target bitrate is 128 kbps per audio channel. Layer-2 is identical with MUSICAM. It has been designed as trade-off between sound quality per bitrate and encoder complexity. It is most useful for bitrates around the "medium" bitrates of 128 or even 96 kbps per audio channel. The DAB (EU 147) proponents have decided to use Layer-2 in the future Digital Audio Broadcasting network. Layer-3: Its ISO target bitrate is 64 kbps per audio channel. Layer-3 merges the best ideas of MUSICAM and ASPEC. It has been designed for best performance at "low" bitrates around 64 kbps or even below. The Layer-3 format specifies a set of advanced features that all address one goal: to preserve as much sound quality as possible even at rather low bitrates. Today, Layer-3 is already in use in various telecommunication networks (ISDN, satellite links, and so on) and speech announcement systems. Q. So how does MPEG audio work? A. Well, first you need to know how sound is stored in a computer. Sound is pressure differences in air. When picked up by a microphone and fed through an amplifier this becomes voltage levels. The voltage is sampled by the computer a number of times per second. For CD audio quality you need to sample 44100 times per second and each sample has a resolution of 16 bits. In stereo this gives you 1,4Mbit per second and you can probably see the need for compression. To compress audio MPEG tries to remove the irrelevant parts of the signal and the redundant parts of the signal. Parts of the sound that we do not hear can be thrown away. To do this MPEG Audio uses psychoacoustic principles. Q. Tell me more about sound quality. How good is MPEG audio compression? And how do you assess that? A. Today, there is no alternative to expensive listening tests. During the ISO-MPEG-1 process, 3 international listening tests have been performed, with a lot of trained listeners, supervised by Swedish Radio. They took place in 7.90, 3.91 and 11.91. Another international listening test was performed by CCIR, now ITU-R, in 92. All these tests used the "triple stimulus, hidden reference" method and the so-called CCIR impairment scale to assess the audio quality. The listening sequence is "ABC", with A = original, BC = pair of original / coded signal with random sequence, and the listener has to evaluate both B and C with a number between 1.0 and 5.0. The meaning of these values is: 5.0 = transparent (this should be the original signal) 4.0 = perceptible, but not annoying (first differences noticable) 3.0 = slightly annoying 2.0 = annoying 1.0 = very annoying With perceptual codecs (like MPEG audio), all traditional parameters (like SNR, THD+N, bandwidth) are especially useless. Fraunhofer-IIS (among others) works on objective quality assessment tools, like the NMR meter (Noise-to-Mask-Ratio), too. If you need more informations about NMR, please contact nmr@iis.fhg.de Q. Now that I know how to assess quality, come on, tell me the results of these tests. A. Well, for details you should study one of those AES papers listed below. One main result is that for low bitrates (60 or 64 kbps per channel, i.e. a compression ratio of around 12:1), Layer-2 scored between 2.1 and 2.6, whereas Layer-3 scored between 3.6 and 3.8. This is a significant increase in sound quality, indeed! Furthermore, the selection process for critical sound material showed that it was rather difficult to find worst-case material for Layer-3 whereas it was not so hard to find such items for Layer-2. For medium and high bitrates (120 kbps or more per channel), Layer-2 and Layer-3 scored rather similar, i.e. even trained listeners found it difficult to detect differences between original and reconstructed signal. Q. So how does MPEG achieve this compression ratio? A. Well, with audio you basically have two alternatives. Either you sample less often or you sample with less resolution (less than 16 bit per sample). If you want quality you can't do much with the sample frequency. Humans can hear sounds with frequencies from about 20Hz to 20kHz. According to the Nyquist theorem you must sample at least two times the highest frequency you want to reproduce. Allowing for imperfect filters, a 44,1kHz sampling rate is a fair minimum. So you either set out to prove the Nyquist theorem is wrong or go to work on reducing the resolution. The MPEG committee chose the latter. Now, the real reason for using 16 bits is to get a good signal-to-noise (s/n) ratio. The noise we're talking about here is quantization noise from the digitizing process. For each bit you add, you get 6dB better s/n. (To the ear, 6dBu corresponds to a doubling of the sound level.) CD-audio achieves about 90dB s/n. This matches the dynamic range of the ear fairly well. That is, you will not hear any noise coming from the system itself (well, there is still some people arguing about that, but lets not worry about them for the moment). So what happens when you sample to 8 bit resolution? You get a very noticeable noise floor in your recording. You can easily hear this in silent moments in the music or between words or sentences if your recording is a human voice. Waitaminnit. You don't notice any noise in loud passages, right? This is the masking effect and is the key to MPEG Audio coding. Stuff like the masking effect belongs to a science called psycho-acoustics that deals with the way the human brain perceives sound. And MPEG uses psychoacoustic principles when it does its thing. Q. Explain this masking effect. A. OK, say you have a strong tone with a frequency of 1000Hz. You also have a tone nearby of say 1100Hz. This second tone is 18 dB lower. You are not going to hear this second tone. It is completely masked by the first 1000Hz tone. As a matter of fact, any relatively weak sounds near a strong sound is masked. If you introduce another tone at 2000Hz also 18 dB below the first 1000Hz tone, you will hear this. You will have to turn down the 2000Hz tone to something like 45 dB below the 1000Hz tone before it will be masked by the first tone. So the further you get from a sound the less masking effect it has. The masking effect means that you can raise the noise floor around a strong sound because the noise will be masked anyway. And raising the noise floor is the same as using less bits and using less bits is the same as compression. Do you get it? Q. I don't get it. A. Well, let me try to explain how the MPEG Audio Layer-2 encoder goes about its thing. It divides the frequency spectrum (20Hz to 20kHz) into 32 subbands. Each subband holds a little slice of the audio spectrum. Say, in the upper region of subband 8, a 6500Hz tone with a level of 60dB is present. OK, the coder calculates the masking effect of this sound and finds that there is a masking threshold for the entire 8th subband (all sounds w. a frequency...) 35dB below this tone. The acceptable s/n ratio is thus 60 - 35 = 25 dB. The equals 4 bit resolution. In addition there are masking effects on band 9-13 and on band 5-7, the effect decreasing with the distance from band 8. In a real-life situation you have sounds in most bands and the masking effects are additive. In addition the coder considers the sensitivity of the ear for various frequencies. The ear is a lot less sensitive in the high and low frequencies. Peak sensivity is around 2 - 4kHz, the same region that the human voice occupies. The subbands should match the ear, that is each subband should consist of frequencies that have the same psychoacoustic properties. In MPEG Layer 2, each subband is 750Hz wide (with 48 kHz sampling frequency). It would have been better if the subbands were narrower in the low frequency range and wider in the high frequency range. That is the trade-off Layer-2 took in favour of a simpler approach. Layer-3 has a much higher frequency resolution (18 times more) - and that is one of the reasons why Layer-3 has a much better low bitrate performance than Layer-2. But there is more to it. I have explained concurrent masking, but the masking effect also occurs before and after a strong sound (pre- and postmasking). Q. Before? A. Yes, if there is a significant (30 - 40dB ) shift in level. The reason is believed to be that the brain needs some processing time. Premasking is only about 2 to 5 ms. The postmasking can be up till 100ms. Other bit-reduction techniques involve considering tonal and non-tonal components of the sound. For a stereo signal you may have a lot of redundancy between channels. All MPEG Layers may exploit these stereo effects by using a "joint- stereo" mode, with a most flexible approach for Layer-3. Furthermore, only Layer-3 further reduces the redundancy by applying huffmann coding. Q. What are the downside? A. The coder calculates masking effects by an iterative process until it runs out of time. It is up to the implementor to spend bits in the least obtrusive fashion. For Layer 2 and Layer 3, the encoder works on 24 ms of sound (with 1152 sample, and fs = 48 kHz) at a time. For some material, the time-window can be a problem. This is normally in a situation with transients where there are large differences in sound level over the 24 ms. The masking is calculated on the strongest sound and the weak parts will drown in quantization noise. This is perceived as a "noise- echo" by the ear. Layer 3 addresses this problem specifically by using a smaller analysis window (4 ms), if the encoder encounters an "attack" situation. Q. Tell me about the complexity. What are the hardware demands? A. Alright. First, we have to separate between decoder and encoder. Remember: the MPEG coding is done asymmetrical, with a much larger workload on the encoder than on the decoder. For a stereo decoder, variuos real-time implementations exist for Layer-2 and Layer-3. They are either based on single-DSP solutions or on dedicated MPEG audio decoder chips. So you need not worry about decoder complexity. For a stereo Layer-2-encoder, various DSP based solutions with one or more DSPs exist (with different quality, also). For a stereo Layer-3-encoder achieving ISO reference quality, the current real-time implementations use two DSP32C and two DSP56002. Q. How many audio channels? A. MPEG-1 allows for two audio channels. These can be either single (mono), dual (two mono channels), stereo or joint stereo (intensity stereo (Layer-2 and Layer-3) or m/s- stereo (Layer-3 only)). In normal (l/r) stereo one channel carries the left audio signal and one channel carries the right audio signal. In m/s stereo one channel carries the sum signal (l+r) and the other the difference (l-r) signal. In intensity stereo the high frequency part of the signal (above 2kHz) is combined. The stereo image is preserved but only the temporal envelope is transmitted. In addition MPEG allows for pre-emphasis, copyright marks and original/copy marks. MPEG-2 allows for several channels in the same stream. Q. What about the audio codec delay? A. Well, the standard gives some figures of the theoretical minimum delay: Layer-1: 19 ms (<50 ms) Layer-2: 35 ms (100 ms) Layer-3: 59 ms (150 ms) The practical values are significantly above that. As they depend on the implementation, exact figures are hard to give. So the figures in brackets are just rough thumb values. Yes, for some applications, a very short delay is of critical importance. E.g. in a feedback link, a reporter can only talk intelligibly if the overall delay is below around 10 ms. If broadcasters want to apply MPEG audio coding, they have to use "N-1" switches in the studio to overcome this problem (or appropriate echo-cancellers) - or they have to forget about MPEG at all. But with most applications, these figures are small enough to present no extra problem. At least, if one can accept a Layer- 2 delay, one can most likely also accept the higher Layer-3 delay. Q. OK, I am hooked on! Where can I find more technical informations about MPEG audio coding, especially about Layer- 3? A. Well, there is a variety of AES papers, e.g. K. Brandenburg, G. Stoll, ...: "The ISO/MPEG-Audio Codec: A Generic Standard for Coding of High Quality Digital Audio", 92nd AES, Vienna 1992, pp.3336 E. Eberlein, H. Popp, ...: "Layer-3, a Flexible Coding Standard", 94th AES, Berlin 93, pp.3493 K. Brandenburg, G. Zimmer, ...: "Variable Data-Rate Recording on a PC Using MPEG-Audio Layer-3", 95th AES, New York 93 B. Grill, J. Herre,... : "Improved MPEG-2 Audio Multi-Channel Encoding", 96th AES, Amsterdam 94 And for further informations, please contact layer3@iis.fhg.de Q. Where can I get more details about MPEG audio? A. Still more details? No shit. You can get the full ISO spec from Omnicom. The specs do a fairly good job of obscuring exactly how these things are supposed to work... Jokes aside, there are no description of the coder in the specs. The specs describes in great detail the bitstream and suggests psychoacoustic models. Originally written by Morten Hjerde <100034,663@compuserve.com>, modified and updated by Harald Popp (layer3@iis.fhg.de). Harald Popp Audio & Multimedia ("Music is the *BEST*" - F. Zappa) Fraunhofer-IIS-A, Weichselgarten 3, D-91058 Erlangen, Germany Phone: +49-9131-776-340 Fax: +49-9131-776-399 email: popp@iis.fhg.de ------------------------------------------------------------------------------- I.3 | MPEG-2 ------------- From: Chad Fogg <cfogg@ole.cdac.com> Date: Tue, 12 Oct 1993 06:23:40 -0700 Subject: installment 2 (posted version) OK: slapped together for your entertainment, it's the second draft installment of the long promised MPEG-2 FAQ. This draft is about 50% complete. Typos or spelling errors have not been checked yet. Many details need to be flushed out. If you have any additional questions or information you would like added, please E-mail to: cfogg@cdac.com ------------------------------------------------------------------------------- [ A short insert ... maybe important for some ... ] From: Tom Pfeifer <pfeifer@fokus.gmd.de> Date: Fri, 29 Apr 1994 16:26:01 +0200 Subject: mpeg2 Heres the number of the MPEG-2 commission draft: Workgroup ISO/IEC JTC 1 SC29N 660 Standard ISO-CD 13818 - {1,2,3} (like usual {system, video, audio}) ------------------------------------------------------------------------------- [ And thats from Chad Fogg again ... ] Table of questions: [near 64KB limit... to big to include in installment 2] Herein is not the official opionions of the MPEG "committee" members. (MPEG opinions are self-cancelling---linear superposition theory). Q. What are the important themes of MPEG video? A. [Other than those introduced by Mark Adler...] 1. Application specific. MPEG does not solve everybody's application needs, but offers a syntax that is a good solution for most. MPEG does not, for example, decorrelate energies situated 1/256th of a pixel between a non-linear combination of 1000 frames. The syntax was designed to occupy an optimum between cost and quality ... in other words, between computational complexity (VLSI area, memory size and bandwidth) and compaction (compression) efficiency. 2. The DCT and Huffman algorithms are some of the least significant aspects of the standard, and yet somehow receive the most press coverage. MPEG-2 made its greatest improvements through enhancement of prediction. 3. In the encoding algorithm, you can do what you want as long as the bistreams produced are compliant. There is a huge difference in picture quality between, for example, the test model and real-world propriety implementions of encoding. Q. Can MPEG-1 encode higher sample rates than 352 x 240 x 30 Hz ? A. Yes. The MPEG-1 syntax permits sampling dimensions as high as 4095 x 4095 x 60 frames per second. The MPEG most people think of as "MPEG-1" is actually a kind of subset known as Constrained Parameters Bitstream (CPB). Q. What are Constrained Parameters Bitstreams? A. CPB are a limited set of sampling and bitrate parameters designed to normalize computational complexity, buffer size, and memory bandwidth while still addressing the widest possible range of applications. CPB limits video to 396 macroblocks (101,376 pixels) per frame if the frame rate is less than or equal to 25 fps (frames per second), and 330 macroblocks (84,480 pixels) per frame if the frame rate is less or equal to 30 fps. Therefore, MPEG video is typically coded at SIF dimensions (352 x 240 x 30fps or 352 x 288 x 25 fps). The total maximum sampling rate is 3.8 Ms/s (million samples/sec) including chroma. The coded video rate is limited to 1.862 Mbit/sec. In industrial practice, the bitrate is the most often waived parameter of CPB, with rates as high as 6 Mbit/sec in use. Q. Why is Constrained Parameters so important? A. It is an optimum point that allows (just barely) cost effective VLSI implementations in 1992 technology (0.8 microns). It also implies a nominal guarantee of interoperability for decoders and encoders. MPEG devices which are not capable of meeting SIF rates are not canonically considered to be true MPEG. Q. Are there ways of getting around constrained parameters bitstreams for SIF class applications and decoders ? A. Yes, some. Remember that CPB limits frames to 396 macroblocks (as in 352 x 288 SIF frames). 416 x 240 x 24 Hz sampling rates are still within the constraints, but this only aids NTSC (240 lines/field) displays. Deviating from 352 samples/line could throw off many decoder implementations that have limited horizontal sample rate conversion modes. Due to chip die size constraints (most chips barely pack in the neccessary features), many decoders use simple doubling, e.g. 352 to 704 samples/line via binary taps which are simple shift-and-add operations. Future MPEG decoders will have arbitrary sample rate convertors on-chip. Also remember that the 1.86 Mbit/sec limit is often ignored in real life. Q. What is MPEG-2 Video Main Profile and Main Level? A. MPEG-2 Video Main Level is analogous to MPEG-1's CPB, with sampling limits at CCIR 601 parameters (720 x 480 x 30 Hz). Profiles limit syntax (i.e. algorithms), whereas Levels limit parameters (sample rates, frame dimensions, coded bitrates, etc.). Together, Video Main Profile and Main Level (abbreviated as MP@ML) normalize complexity within feasible limits of 1994 VLSI technology (0.5 micron), yet still meet the needs of the majority of application users. Level Max. sampling Pixels/ Max. Significance dimensions fps sec bitrate --------- ---------------- ------- ------- -------------------------- Low 352 x 240 x 30 3.05 M 4 Mb/s CIF, consumer tape equiv. Main 720 x 480 x 30 10.40 M 15 Mb/s CCIR 601, studio TV High 1440 1440 x 1152 x 30 47.00 M 60 Mb/s 4x 601, consumer HDTV High 1920 x 1080 x 30 62.70 M 80 Mb/s production SMPTE 240M std Note 1: pixel rate and luminance (Y) sample rate are equivalent. 2: Low Level is similar MPEG-1's Constrained Parameters Bitstreams. Profile Comments ------- ----------------------------------------------------------- Simple Same as Main, only without B-pictures. Intended for software applications, perhaps CATV. Main Most decoder chips, CATV, satellite. 95% of users. Main+ Main with Spatial and SNR scalability Next Main+ with 4:2:2 marcoblocks Profile Level Simple Main Main+ Next ------------ -------------- -------------- -------------- ------------ High illegal illegal 4:2:2 chroma High-1440 illegal With spatial 4:2:2 chroma Scalablity Main 90% of users Main with SNR 4:2:2 chroma scalability Low illegal Main with SNR illegal scalabiliy [Subject to change at whim of MPEG Requirements sub-group] Q. How do you tell a MPEG-1 bitstream from a MPEG-2 bistream? A. All MPEG-2 bistreams must have certain extension headers that *immediately* follow MPEG-1 headers. At the highest layer, END ---------------------- CUT HERE --------------------- 1/6