ftp.cdrom.com/pub/cdrom//cdrom.tar/cdrom/textinfo/cd_intro.txt

Text File | 1993-02-10 | 24KB | 497 lines

Article 3273 in alt.cd-rom:
From: [e-mail] (Harvey Grosser)
Subject: Here's the format of data on CDs and CD-ROMs
Message-ID: <[e-mail]>
Date: 4 Jun 92 08:03:24 GMT
Sender: CD-ROM <[e-mail]>
Reply-To: Harvey Grosser <[e-mail]>
Distribution: alt
Organization: FUNET-NGW
Lines: 478

The following document is available on Apple's FTP site ftp.apple.com
as "pub/cd-rom/cd-rom.summary".  I think you'll find it very interesting.

                Harvey Grosser
                [e-mail]
--------------------------------------------
The following is part 1 of a nice intro to CD technology written
by Andy Poggio in 1988.  It appears here in its original form.

CD Summary Introduction

As requested by many people, I will post this CD Summary over the next
several days in five parts of which this is the first.  I received
requests from rec.audio, comp.ivideodisc, and comp.graphics -- so I will
post it to all these groups.  I'm not sure that it is appropriate for
comp.graphics but I DID receive multiple requests to post it there.

The summary is somewhat technical but more important it is factual: I
wrote it after reading the original CD standards documents available from
Sony or Philips to CD licensees.  If you are interested in the standards
documents, you need to contact them directly -- sorry, I don't have a
specific contact or phone number.

I do work for Apple but this summary contains a minimum of Apple
references.  I hope everyone agrees that the result is in keeping with net
policy on the matter.

--andy

CD Summary Part 1

CD-ROM Technical Summary
>From Plastic Pits to "Fantasia"


                                               Andy Poggio
                                               March, 1988


Abstract

This summary describes how information is encoded on Compact Disc (CD)
beginning with the physical pits and going up through higher levels of
data encoding to the structured multimedia information that is possible
with programs like HyperCard.  This discussion is much broader than any
single standards document, e.g. the CD-Audio Red Book, while omitting much
of the detail needed only by drive manufacturers.

Salient Characteristics

1.  High information density -- With the density achievable using optical
encoding, the CD can contain some 540 megabytes of data on a disc less
than five inches in diameter.

2.  Low unit cost -- Because CDs are manufactured by a well-developed
process similar to that used to stamp out LP records, unit cost in large
quantities is less than two dollars.

3.  Read only medium -- CD-ROM is read only; it cannot be written on or
erased.  It is an electronic publishing, distribution, and access medium;
it cannot replace magnetic disks.

4.  Modest random access performance -- Due to optical read head mass and
data encoding methods, random access ("seek time") performance of CD is
better than floppies but not as good as magnetic hard disks.

5.  Robust, removable medium -- The CD itself is comprised mostly of, and
completely coated by, durable plastic.  This fact and the data encoding
method allow the CD to be resistant to scratches and other handling
damage.  Media lifetime is expected to be long, well beyond that of
magnetic media such as tape.  In addition, the optical servo scanning
mechanism allows CDs to be removed from their drives.

6.  Multimedia storage -- Because all CD data is stored digitally, it is
inherently multimedia in that it can store text, images, graphics, sound,
and any other information expressed in digital form.  Its only limit in
this area is the rate at which data can be read from the disc, currently
about 150 KBytes/second.  This is sufficient for all but uncompressed,
full motion color video.
CD Summary Part 2

CD  Data  Hierarchy

Storing data on a CD may be thought of as occurring through a data
encoding hierarchy with each level built upon the previous one.  At the
lowest level, data is physically stored as pits on the disc.  It is
actually encoded by several low-level mechanisms to provide high storage
density and reliable data recovery.  At the next level, it organized into
tracks which may be digital audio or CD-ROM.  The High Sierra
specification then defines a file system built on CD-ROM tracks.  Finally,
applications like HyperCard specify a content format for files.

The Physical Medium

The Compact Disc itself is a thin plastic disk some 12 cm. in diameter.
Information is encoded in a plastic-encased spiral track contained on the
top of the disk.  The spiral track is read optically by a noncontact head
which scans approximately radially as the disk spins just above it.  The
spiral is scanned at a constant linear velocity thus assuring a constant
data rate.  This requires the disc to rotate at a decreasing rate as the
spiral is scanned from its beginning near the center of the disc to its
end near the disc circumference.

The spiral track contains shallow depressions, called pits, in a
reflective layer.  Binary information is encoded by the lengths of these
pits and the lengths of the areas between them, called land.  During
reading, a low power laser beam from the optical head is focused on the
spiral layer and is reflected back into the head.  Due to the optical
characteristics of the plastic disc and  the wavelength of light used, the
quantity of reflected light varies depending on whether the beam is on
land or on a pit.  The modulated, reflected light is converted to a radio
frequency, raw data signal by a photodetector in the optical head.

Low-level Data Encoding

To ensure accurate recovery, the disc data must be encoded to optimize the
analog-to-digital conversion process that  the radio frequency signal must
undergo.  Goals of the low level data encoding include:

1.  High information density.  This requires encoding that makes the best
possible use of the high, but limited, resolution of the laser beam and
read head optics.

2.  Minimum intersymbol interference.  This requires making the minimum
run length, i.e. the minimum number of consecutive zero bits or one bits,
as large as possible.

3.  Self-clocking.  To avoid a separate timing track, the data should be
encoded so as to allow the clock signal to be regenerated from the data
signal.  This requires limiting the maximum run length of the data so that
data transitions will regenerate the clock.

4.  Low digital sum value (the number of one bits minus the number of zero
bits).  This minimizes the low frequency and DC content of the data signal
which permits optimal servo system operation.

A straightforward encoding would be to simply to encode zero bits as land
and one bits as pits.  However, this does not meet goal (1) as well as the
encoding scheme actually used.  The current CD scheme encodes one bits as
transitions from pit to land or land to pit and zero bits as constant pit
or constant land.

To meet goals (2) to (4), it is not possible to encode arbitrary binary
data.  For example, the integer 0 expressed as thirty-two bits of zero
would have too long a run length to satisfy goal (3).  To accommodate
these goals, each eight-bit byte of actual data is encoded as fourteen
bits of channel data.  There are many more combinations of fourteen bits
(16,384) than there are of eight bits (256).  To encode the eight-bit
combinations, 256 combinations of fourteen bits are chosen that meet the
goals.  This encoding is referred to as Eight-to-Fourteen Modulation (EFM)
coding.

If fourteen channel bits were concatenated with another set of fourteen
channel bits, once again the above goals may not be met.  To avoid this
possibility, three merging bits are included between each set of fourteen
channel bits.  These merging bits carry no information but are chosen to
limit run length, keep data signal DC content low, etc.  Thus, an eight
bit byte of actual data is encoded into a total of seventeen channel bits:
 fourteen EFM bits and three merging bits.

To achieve a reliable self-clocking system, periodic synchronization is
necessary.  Thus, data is broken up into individual frames each beginning
with a synchronization pattern.  Each frame also contains twenty-four data
bytes, eight error correction bytes, a control and display byte (carrying
the subcoding channels), and merging bits separating them all.  Each frame
is arranged as follows:

Sync Pattern24 + 3channel bits
Control and Display byte14 + 3
Data bytes12 * (14 + 3)
Error Correction bytes 4 * (14 + 3)
Data bytes12 * (14 + 3)
Error Correction bytes 4 * (14 + 3)

TOTAL588channel bits

Thus, 192 actual data bits (24 bytes) are encoded as 588 channel bits.

Editorial:  A CD physically has a single spiral track about 3 miles long.
CDs spin at about 500 RPM when reading near the center down to about 250
RPM when reading near the circumference.

Disc with a 'c' or disk with a 'k'?  A usage has emerged for these terms:
disk is used for eraseable disks (e.g. magnetic disks) while disc is used
for read-only (e.g. CD-ROM discs).  One would presumably call a frisbee a
disc.

--andy
CD Summary Part 3

First Level Error Correction

Data errors can arise from production defects in the disk itself, defects
arising from subsequent damage to the disk, or jarring during reading.  A
significant characteristic of  these errors is that they often occur in
long bursts.  This could be due, for example, to a relatively wide mark on
the disc that is opaque to the laser beam used to read the disc.  A system
with two logical components called the Cross Interleave Reed-Solomon
Coding (CIRC) is employed for error correction.  The cross interleave
component breaks up the long error bursts into many short errors; the
Reed-Solomon component provides the error correction.

As each frame is read from the disc, it is first decoded from fourteen
channel bits (the three merging bits are ignored) into eight-bit data
bytes.  Then, the bytes from each frame (twenty-four data bytes and eight
error correction bytes) are passed to the first Reed-Solomon decoder which
uses four of the error correction bytes and is able to correct one byte in
error out of the 32.  If there are no uncorrectable errors, the data is
simply passed along.  If there are errors, the data is marked as being in
error at this stage of decoding.

The twenty-four data bytes and four remaining error correction bytes are
then passed through unequal delays before going through another
Reed-Solomon decoder.  These unequal delays result in an interleaving of
the data that spreads long error bursts among many different passes
through the second decoder.  The delays are such that error bursts up to
450 bytes long can be completely corrected.  The second Reed-Solomon
decoder uses the last four error correction bytes to correct any remaining
errors in the twenty-four data bytes.  At this point, the data goes
through a de-interleaving process to restore the correct byte order.

Subcoding Channels and Blocks

The eight-bit control and display byte in each frame carries the subcoding
channels.  A subcoding block consists of 98 subcoding bytes, and thus 98
of the 588-bit frames.  A block then can contain 2352 bytes of data.
Seventy-five blocks are read each second.  With this information, it is
now straightforward to calculate that the CD data rate is in fact correct
for CD digital audio (CD-DA):

Required CD digital audio data rate:  44.1 K samples per second * 16 bits
per sample * 2 channels = 1,411,200 bits/sec.

CD data rate:  8 bits per byte * 24 bytes per frame * 98 frames per
subcoding block * 75 subcoding blocks per second = 1,411,200 bits/sec.

The eight subcoding channels are labeled P through W and are encoded one
bit for each channel in a control and display byte.  Channel P is used as
a simple music track separator.  Channel Q is used for control purposes
and encodes information like track number, track type, and location
(minute, second, and frame number).  During the lead-in track of the disc,
channel Q encodes a table of contents for the disk giving track number and
starting location.  Standards have been proposed that would use the
remaining channels for line graphics and ASCII character strings, but
these are seldom used.

Track Types

Tracks can have two types as specified in the control bit field of
subchannel Q.  The first type is CD digital audio (CD-DA) tracks.  The
two-channel audio is sampled at 44.1 Khz with sixteen bit linear sampling
encoded as twos complement numbers.  The sixteen bit samples are separated
into two eight-bit bytes; the bytes from each channel alternate on the
disc.   Variations for audio tracks include pre-emphasis and four track
recording.

The other type of track specified by the subchannel Q control bit field is
the data track.  These must conform to the CD-ROM standard described
below.  In general, a disc can have a mix of CD digital audio tracks and a
CD-ROM track, but the CD-ROM track must come first.

Editorial:  This first level error correction (the only type used for CD
Audio data) is extremely powerful.  The CD specification allows for discs
to have up to 220 raw errors per second.  Every one of these errors is
(almost always) perfectly corrected by the CIRC scheme for a net error
rate of zero.  For example, our tests using Apple's CD-ROM drive (which
also plays audio) show that raw error rates are around 50-100 per second
these days.  Of course, these are perfectly corrected, meaning that the
original data is perfectly recovered.  We have tested flawed discs with
raw rates up to 300 per second.  Net errors on all of these discs?  Zero!
I would expect a typical audio CD player to perform similarly.  Thus I
expect this raw error rate to have no audible consequences.

So why did I say "almost always" corrected above?  Because a sufficiently
bad flaw may produce uncorrectable errors.  These very unusual errors are
"concealed" by the player rather than corrected.  Note that this
concealment is likely to be less noticeable than even a single scratch on
an LP.  Such a flaw might be a really opaque finger smudge; CDs do merit
careful handling.  On the two (and only two) occasions I have found these,
I simply sprayed on a little Windex glass cleaner and wiped it off using
radial strokes.  This restored the CDs to zero net errors.

One can argue about the quality of the process of conversion of analog
music to and from digital representation, but in the digital domain CDs
are really very, very good.

CD Summary Part 4

CD-ROM Data Tracks

Each CD-ROM data track is divided into individually addressable blocks of
2352 data bytes, i.e. one subcoding block or 98 frames.  A header in each
block contains the block address and the mode of the block.  The block
address is identical to the encoding of minute, second, and frame number
in subcode channel Q.  The modes defined in the CD-ROM specification are:

Mode 0 -- all data bytes are zero.

Mode 1 -- (CD-ROM Data):
Sync Field - 12 bytes
Header Field - 4
User Data Field - 2048
Error Detection Code - 4
Reserved - 8
Error Correction - 276

Mode 2 -- (CD Audio or Other Data):
Sync Field - 12 bytes
Header Field - 4
User Data Field - 2048
Auxiliary Data Field - 288

Thus, mode 1 defines separately addressable, physical 2K byte data blocks
making CD-ROM look at this level very similar to other digital mass
storage devices.

Second Level Error Correction

An uncorrected error in audio data typically results in a brief, often
inaudible click during listening at worst.  An uncorrected error in other
kinds of data, for example program code, may render a CD unusable.  For
this reason, CD-ROM defines a second level of error detection and error
correction (EDC/ECC) for mode 1 data.  The information for the EDC/ECC
occupies most of the auxiliary data field.

The error detection code is a cyclic redundancy check (CRC) on the sync,
header, and user data.  It occupies the first four bytes of the auxiliary
data field and provides a very high probability that uncorrected errors
will be detected.  The error correction code is essentially the same as
the first level error correction in that interleaving and Reed-Solomon
coding are used.  It occupies the final 276 bytes of the auxiliary data
field.

Editorial:  This extra level of error correction for CD-ROM blocks is one
of the many reasons that CD-ROM drives are much more expensive than
consumer audio players.  To perform this error correction quickly requires
substantial extra computing power (sometimes a dedicated microprocessor)
in the drive.

This is also one reason that consumer players like the Magnavoxes which
claim to be CD-ROM compatible (with their digital output jack on the back)
are useless for that purpose.  They have no way of dealing with the CD-ROM
error correction.  They also have no way for a computer to tell them where
to seek.

Another reason that CD-ROM drives are more expensive is that they are
built to be a computer peripheral rather than a consumer device, i.e. like
a combination race car/truck rather than a family sedan.  One story,
probably apocryphal but not far from the truth, has it that a major
Japanese manufacturer tested some consumer audio players to simulate
computer use:  they made them seek (move the optical head) from the inside
of the CD to the outside and back again.  These are called maximum seeks.
The story says they managed to do this for about 24 hours before they
broke down.  A CD-ROM drive needs to be several orders of magnitude more
robust.  Fast and strong don't come cheap.
CD Summary Part 5

The High Sierra File System Standard

Built on top of the addressable 2K blocks that the CD-ROM specification
defines, the next higher level of data encoding is a file system that
permits logical organization of the data on the CD.  This can be a native
file system like the Macintosh Hierarchical File System (HFS).  Another
alternative is the High Sierra (also known as the ISO 9660) file standard,
recently approved by the National Information Standards Organization
(NISO) and the International Standards Organization (ISO), which defines a
file system carefully tuned to CD characteristics.  In particular:

1.  CDs have modest seek time and high capacity.  As a result, the High
Sierra standard makes tradeoffs that reduce the number of seeks needed to
read a file at the expense of space efficiency.

2.  CDs are read-only.  Thus, concerns like space allocation, file
deletion, and the like are not addressed in the specification.

For High Sierra file systems, each individual CD is a volume.  Several CDs
may be grouped together in a volume set and there is a mechanism for
subsequent volumes in a set to update preceding ones.  Volumes can contain
standard file structures, coded character set file structures for
character encoding other than ASCII, or boot records.  Boot records can
contain either data or program code that may be needed by systems or
applications.

High Sierra Directories and Files

The file system is a hierarchical one in which directories may contain
files or other directories.  Each volume has a root directory which serves
as an ancestor to all other directories or files in the volume.  This
dictates an overall tree structure for the volume.

A typical disadvantage in hierarchical systems is that to read a file
(which must be a leaf of the hierarchy tree) given its full path name, it
is necessary to begin at the root directory and search through each of its
ancestral directories until the entry for the file is found.  For example,
given the path name "Wine Regions:America:California:Mendocino", three
directories (the first three components of the path name) would need to be
searched.  Typically, a separate seek would be required for each
directory.  This would result in relatively poor performance.

To avoid this, High Sierra specifies that each volume contain a path table
in addition to its directories and files.  The path table describes the
directory hierarchy in a compact form that may be cached in computer
memory for optimum performance.  The path table contains entries for the
volume's directories in a breadth-first order; directories with a common
parent are listed in lexicographic order.  Each entry contains only the
location of the directory it describes, its name, and the location in the
path table of its parent.  This mechanism allows any directory to be
accessed with only a single CD seek.

Directories contain more detailed information than the path table.  Each
directory entry contains:

Directory or file location.
File length.
Date and time of creation.
Name of the file.
Flags:
Whether the entry is for a file or a directory.
Whether or not it is an associated file.
Whether or not it has records.
Whether or not it has read protection.
Whether or not it has subsequent extents.
Interleave structure of the file.

Interleaving may be used, for example, to meet realtime requirements for
multiple files whose contents must be presented simultaneously.  This
would happen if a file containing graphic images were interleaved with a
file containing compressed sound that describes the images.

Files themselves are recorded in contiguous (or interleaved) blocks on the
disc.  The read-only nature of CD permits this contiguous recording in a
straightforward manner.  A file may also be recorded in a series of
noncontiguous extents with a directory entry for each extent.

The specification does not favor any particular computer architecture.  In
particular all significant, multibyte numbers are recorded twice, once
with the most significant byte first and once with the least significant
byte first.

Multimedia Information

Using the file system are applications that create and portray multimedia
information.  While it is true that a CD can store anything that a
magnetic disk can store (and usually much more of it), CDs will be used
more for storing information than for storing programs.  It is the very
large storage capacity of CDs coupled with their low cost that opens up
the possibilities for interactive, multimedia information to be used in a
multitude of ways.

Programs like HyperCard, with it's ease of authoring and broad
extensibility, are very useful for this purpose.  Hypercard stacks, with
related information such as color images and sound, can be easily and
inexpensively stored on CDs despite their possibly very large size.

Editorial:  The High Sierra file system gets its name from the location of
the first meeting on it:  the High Sierra Hotel at Lake Tahoe.  It is much
more commonly referred to as ISO 9660, though the two specifications are
slightly different.

It has gotten very easy and inexpensive to make a CD-ROM disc (or audio
CD).  For example, you can now take a Macintosh hard disk and send it with
$1500 to one of several CD pressers.  They will send you back your hard
disk and 100 CDs with exactly the same content as what's on your disk.
This is the easy way to make CDs with capacity up to the size of your hard
disk (Apple's go up to 160 megabytes).  True, this is not a full CD but
CDs don't need to be full.  If you have just 10 megabytes and need 100
copies, CDs may be the best way to go.

If you are buying a CD-ROM drive, there are several factors you might
consider in making your choice.  Two factors NOT to consider are capacity
and data rate.  The capacity of all CD-ROM drives is determined solely by
the CD they are reading.  Though you will see a range of numbers in
manufacturers' specs (e.g. 540, 550, 600, and 650 Mbytes), any drive can
read any disc and so they are all fundamentally the same.  All CD-ROM
drives read data at a net 150 Kbytes/sec for CD-ROM data.  Other data
rates you may see may include error correction data (not included in the
net rate) or may be a mode 2 data rate (faster than mode 1).  All drives
will be the same in all of these specs.

[ Editorial:  The last paragraph is wrong on both points.  This may have
been true in 1988, when this article was written, but is no longer true
today (in 1992).  A few drives cannot read CDROMs with more than about
620 megabytes on them.  There is a huge variation in speed.  Today only
the slowest drives read at 150 kb/sec.  Most read at least 300 and at
least one (the Pioneer DRM-600X) reads at 600 kb/sec. ]