NetNews Usenet Archive 1993 #1/NN_1993_1.iso/spool/comp/graphics/visualiz/1979

Internet Message Format | 1993-01-07 | 24.9 KB

Path: sparky!uunet!zaphod.mps.ohio-state.edu!rpi!gatech!usenet.ins.cwru.edu!agate!dog.ee.lbl.gov!network.ucsd.edu!dudley!nadeau
From: [e-mail] (Dave Nadeau,,45062)
Newsgroups: comp.graphics.visualization
Subject: Re: SDSC Image Tools and PBM Plus
Date: 7 Jan 1993 23:43:29 GMT
Organization: San Diego Supercomputer Center @UCSD.
Lines: 473
Distribution: world
Message-ID: <[e-mail]>
References: <[e-mail]>
Reply-To: [e-mail]
NNTP-Posting-Host: dudley.sdsc.edu


In <[e-mail]> [e-mail] (Dave Nadeau,,45062) writes:

> |"1.  The SDSC Image Tools were designed as a C function library rather than
> |"    a set of independent tools, such as the PBMPlus package.  Using a function
> |"    library, image manipulation algorithms are encapsulated into generic
> |"    functions maintained in a central location.  Tool programmers are able to
> |"    quickly write new tools by simply calling library routines.  Tool source
> |"    is kept small and succinct.  If bugs are found in the underlying
> |"    library's image algorithm, the bug is fixed in one place and all tools
> |"    using that routine are relinked.  No tool source modification is required.
>
> PBMPLUS also has libraries that help write new PBMPlus formats; the pbm, pgm,
> ppm and pnm libraries.

Indeed, PBMPLUS does have libraries:  but only for the PBM formats.  File
format knowledge for other formats is distributed into multiple tools.  Image
manipulation functionality is similarly distributed, rather centralized.

It is not possible to write a new tool that outputs non-PBM file formats unless
one copies source from another PBM tool.  Since the format is a constant,
and that source is a constant (except for bug fixes), it would be nice to
put it in a "safe" "central" place.  This is the founding principle of
libraries, of whatever type, and one that has driven the development of the
SDSC Image Tools.  It is a concept that is only partially implemented in the
PBMPLUS tools.

Please read the Image Tools manual pages "imintro" and "ImFileRead" for a
full discussion of data structures and image file format support.  These
man pages are included in the release.


>   [ Programmer supposedly has to clone the code multiple times ]
>
> No. The programmer can take the code from PBMPlus (as source is available) and
> create his own library. Difference is that PBMPlus, as it provides source and
> an easy to manipulate format, leaves it up to the programmer to use either its
> fill function or a programmer supplied one. It is short-sighted to think that
> programmers would not be able to do so. The difference with your package is
> that you have made the decision on what fill algorithm to use, in other words,
> it is you who made the decision to take whatever code and put it in a library,
> thus saving the programmer the trouble.

The programmer using the SDSC Image Tools is free to implement any fill
algorithm they choose using the rich set of data structures and data structure
access routines and macros provided within the SDSC Image Tools.  We have in
no way barred anybody from doing anything they feel like with image data.  We
have merely taken many of the most common actions and centralized them into
the image library.  If a user doesn't like our fill function, they are free
to and encouraged to implement their own.

However, most users do not have the time to re-develope their own image
manipulation routines.  Instead, they'd rather use somebody else's and get
on with the task they really wish to accomplish.  We make this easier by
placing that routines in a library and heavily document their abilities.
Such users are not required to cut-and-paste undocumented sparsely commented
code in order to get the own work done.

Please read the manual page "imintro" in the SDSC Image Tools release for a
start on understanding the internal data structures and access functions.
Alternatively, you may read any of the published papers on the Image Tools
for further discussion of the data structures involved.


> |"2.  All SDSC Image Tools can read and write all of our supported image
> |"    file formats.  This is another direct result of encapsulating file format
> |"    handling within generic library functions.
>
> [ examples of how to do this deleted ]
>
> It also leads to huge executables, when shared libraries are not available;
> even if the program only reads one format ever, it still has the code for all
> the other formats in its executable!

Quite true.  There is no perfect solution (that I am aware of).  We have
chosen to reduce the amount of tools the user must remember for doing image
file format conversion from 87+ to 1.  The increase in binary size is
compensated for by a decrease in user hassle.  The priority is placed on the
user's needs, rather than the disk drive's.


> Keep in mind that PBMPlus is volunteer work; Jef Poskanzer has done a great
> job of creating portable software - due to the fact that the format is simple,
> and easily extendable, a huge amount of conversion tools are available. The
> things you mention are, of course, things to be done, but one lives and
> learns; what you suggest will sure be included in PBMPlus, someday.

Indeed, the PBMPlus package is one everybody should have.  Jef Poskanzer has
done an excellent service to the industry and one we hope he will continue to
provide.  There is no such thing as having too many tools.  If one package
doesn't satisfy your needs, perhaps another will.  We developed the SDSC
Image Tools to satisfy a class of needs that we did not see as being satisfied
by other packages out there.  However, in the same vein, PBMPlus satisfies a
number of needs the Image Tools do not.  You are free to choose whatever
package you wish.  We recommend that you choose both.


> |"    Furthermore, PBMPlus tools that actually do something to images only work
> |"    on PBMPlus file formats.  If you want to read any other kind of image, you
> |"    first have to convert into one of the PBMPlus formats.  Then if you want
> |"    a result in another format, such as for transfer to a Mac or PC, or use
> |"    as a texture map in a ray tracer, you have to convert back out of the
> |"    PBMPlus formats into what you really wanted in the first place.
>
> That's the general idea - rather than dealing with millions of file formats,
> one format that is easy to manipulate is used, thus greatly reducing the
> amount of code needed. The beauty of the package is by introducing an
> intermediary format suddenly the number of converters can be reduced from NxM
> (your case) to N+M. A very basic strategy, which can be found in most
> introductory courses on computer architecture and compiler writing.

The PBMPlus package and the SDSC Image Tools use approximagely the same
computational strategy for developing image file format handlers:  all
conversion is done to and from an intermediary data structure.  PBMPlus
formallizes those data structure as a set of file formats.  The SDSC Image
Tools leave them as memory data structures.  In both cases this results in
N + M translators, not N * M.

    In PBMPlus there are N format read tools that read into PBM formats,
    and M write tools that write out from PBM formats.

    In the Image Tools there are N format read routines that read into
    memory data structures, and M write routines that write out from
    memory data structures.  The N read and M write routines are internal
    to the package and available to the programmer as a single read and
    a single write routine.

The Image Tools eliminate the need for the intermediate file formats.  It seems
that we already have enough file formats out there in the world without
having to invent still more.

Please read the "imfileread" manual page included in the Image Tools release
for full details on reading and writing image file formats.


> |"3. Another natural result of developing the SDSC Image Tools as a C function
> |"   library is the ease with which image file handling can be added to existing
> |"   tools.  Non-SDSC users have built AVS and Explorer modules using the
> |"   SDSC Image Tools library, added image handling support to existing tools,
> |"   written windowing system user interfaces for the tools, and so on.
>
> Great. The same can be done just as easily for PBMPLUS. I have done so; the
> format is so simple, that implementation and use of it is no work at all.

As stated earlier and in-depth in the referenced posting, the PBMPlus libraries
only support reading and writing of the PBM file formats (pbm, pgm, and ppm).
New tools, or modifications of existing tools, can add support for these
formats.  However, adding support for non-PBM file formats requires that source
be copied from PBM tools.  No library calls for non-PBM file format I/O
exist.  Non-PBM file format support is distributed throughout multiple PBMPlus
package tools, and not centralized within a library.

The Image Tools place all file format I/O functionality within a central
library.  Tool authors can read or write any of the support file formats by
making a single call to the library.  They do not have to reimplement the
often very complex quirks of image file formats, and do not have to
cut-and-paste code from other people's code and try to figure out how to make
it work for their own tool.

Of course, if the programmer WANTS to read or write their own image file format,
they can do that with the SDSC Image Tools as well.  The Image Tools do not
prevent programmer activites, they just make common ones easier to do.


> |"    At SDSC, we have implemented a generic lpr-based print filter system that
> |"    drives a number of computer graphics output devices from any of the
> |"    supported image file formats.  For instance, users of the system can queue
> |"    images to be printed to slides on a film recorder using:
> |"
> |"        vpr -Pslides title.rgb frames*.pix credits{1,2,3,4}.ras
> |"
> |"   There are no hardcopy device-specific file formats.  There is no need to
> |"   convert your SGI RGB image to quirky device protocol streams, then
> |"   transfer the file over to the specific host that drives the printer, then
> |"   invoke printer-specific software to print your image.  Instead, vpr handles
> |"   it all.  Images are transferred via lpr to the remote host that drives
> |"   the printer.  The lpr daemon invokes a device-specific print filter that
> |"   reads in the images, using the Image Tools library, adjusts image colors
> |"   appropriately for the device, again using the Image Tools library, and
> |"   scales the image up or down to the device resolution, once again using
> |"   the Image Tools library.  The final image is written directly to the
> |"   device, with communications protocol handled on the spot.
>
> Nice. This can also be done by PBMPLUS, by having the program/driver convert
> to PBMPLUS format upon arrival. No big deal.

While do-able with PBMPlus, it is more difficult because there is no central
intelligence about file formats.  There is no table of information about
file formats and magic numbers, and thus no trivial way to write a tool that
can automatically figure out what PBM tool to invoke to convert an incomming
file format to a PBM format.  This is not an impossible thing to do, but it
is a pain.  This work has already been done in the Image Tools.  A single
function call is all that it takes to determine the file format type of an
image file.

An implementation using PBMPlus would also be slower.  Such a PBMPlus
implementation would intuit the type of the incomming image file (via some
built-in table), then invoke the proper PBMPlus tool by fork-and-exec
or by system().  That tool would read the input file and generate a PBM file
(or pipe stream).  The print spooler would then read in that PBM file (or
stream) and do its thing.  This sequence requires that the image data be read
twice:  once from the original file, and once from a PBM file (or stream).
It also requires that the image data be written once:  to the PBM file (or
stream).

The Image Tools, however, read the input file directly using the library
file read routine.  There is no "extra" disk I/O to a second file (or pipe)
and no second reading of the same data.  The Image Tools will be faster
for the same operation.


> |"  This kind of integration of image file format I/O into existing tools is not
> |"  possible using the PBMPlus package.  File format code is distributed all
> |"  about into multiple PBMPlus tools.  There is no file format function
> |"  library.
>
> There is. Check the manual pages pbm(5),pgm(5),ppm(5) and pnm(5). I'll admit
> that the documentation could use some work, but hey, what do you expect ? It
> is FREE and it has SOURCE.

As already stated, the PBMPlus library supports routines for reading and
writing the PBM formats only.  It does not support reading and writing the
other file formats.  That code is distributed throughout the tool set and
is largely undocumented.

The Image Tools library supports reading and writing of all our supported
image file formats.  Code is centralized, accessible through a single
function call, and fully documented.  Please read the manual page for
"imfileread" for full details on image file I/O within the Image Tools.


> |"4.  The SDSC Image Tools were designed to support a wide range of image
> |"    storage types:  monochrome, grayscale, color indexed (also called
> |"    pseudocolor), and RGB (also called truecolor), and images of all these
> |"    types that have alpha channels, Z-buffers, and other non-visual pixel
> |"    channels.
> |"
> |"    The PBMPlus tools, however, only support monochrome, grayscale, and RGB
> |"    images.  Alpha channels are unsupported and silently stripped from input
> |"    files as you convert into the less-functional PBMPlus file formats.  There
> |"    is no support whatsoever for color indexed images, such as those used by
> |"    non-RGB color graphics displays and most color image file formats.  To
> |"    handle a color indexed image, PBMPlus converts it to RGB.  The color table
> |"    is applied to the color indexes and then tossed.  The "indexed-ness" of
> |"    the original image is lost.  This is unacceptable.
>
> Not really; the original can always be regenerated by functions that are
> present in the various p*m-libraries.

Not true.  The original can *not* always be regenerated.  An 8-bit image can
be recreated from a 24-bit image that was itself originally created from an
8-bit image and visually they will be identical.  The data content, however,
will not be identical.

Consider this example:

    Image 1 is an 8-bit image with the following color table:
        entry 1:    255 0 0
        entry 2:    0 255 0
        entry 3:    0 0 255
        entry 4:    255 0 0
        entry 5:    128 128 128

    Entry's 1 through 4 are used in the image.  Entry 5 is not, but is
    nevertheless part of the data and may have significance to the user
    of the data.

    Entry 4 is identical in color to entry 1, but used in different places
    in the image, and perhaps with different intent.  The border of the
    image might be entry 1 (red), at the same time as a "Danger, Nuclear
    Meltdown Imminent" message uses entry 4 (also red).  The use of these
    two entries is different.  So, while they contain identical colors,
    the use of entry 4 instead of 1 has significance to the user of
    the data.

    The ordering of colors in Image 1's table may have significance to
    the user of the data.  Perhaps the first color is the background,
    and the second the foreground, the third the text color, and so on.
    Or perhaps entry 1 is bone, entry 2 muscle, entry 3 air, and
    so on in a CATscan image.  The ordering of entries in the color table
    maps directly to values obtained from the CATscanner.  The order is
    of significance, independent of the color mapping used.

    Now, convert Image 1 to Image 2, a 24-bit image in the PBMPlus package.
    Each pixel in Image 1 is scanned, its color table entry found and that
    entry's color placed into Image 2's 24-bit pixel.

    Pixels using entry 1 and entry 4 in Image 1 will both end up with the
    same color in Image 2.  The differentness of pixels using these two
    entries has been lost.

    No pixels with the color of entry 5 will be in Image 2.  The existance
    of entry 5 has been lost.

    Now, convert Image 2 to Image 3, a new 8-bit image with a color table.
    The pixels in Image 2 are scanned and a table of unique colors
    generated.  The order of colors in that table is the order of unique
    colors occuring in the image, left to right, top to bottom.  Or perhaps
    the table is sorted afterwards.  It doesn't matter.  The order
    of colors will not be identical to the order in Image 1's color table
    except by chance.

    Conversion into and out of the PBMPlus package has left us with an
    8-bit image that is visually identical to the input 8-bit image, but
    has less information.  The output image has lost:

        1.  Color table entry order significance.
        2.  Duplicate color table entry significance.
        3.  Extra color table entry significance.

    If you are in a situation where none of these matter, the PBMPlus
    package is just fine.  If, however, you depend upon color table entry
    order or associate signficance to duplicate or extra color table
    entries, such as in scientific visualization, then conversion through
    the PBMPlus package has destroyed the validity of some of your data.

    The Image Tools support 8-bit images directly and their use will not
    result in the loss of this data.


> |"    xwdtopnm input.xwd | pnmtoxwd > output.xwd
> |"
> |"    Format readers parse the input file and drop tags and values into the
> |"    tag table.  Format writers scan the tag table for items relevant for the
> |"    output format and write them out.  The XWD reader, for instance, reads
> |"    in the image window name and drops it into the tag table along with the
> |"    image pixels and color table.  The Sun rasterfile RAS writer scans the
> |"    tag table for images and color tables and writes them out, but it ignores
> |"    the image window name because the RAS format does not support image names.
> |"    The XWD writer, on the other hand, does pick out the image name from the
> |"    tag table and write it out to the output format.  It all depends upon the
> |"    abilities and limitations of the output file format.  The SDSC Image Tools
> |"    do not arbitrarily limit image data to those things that fit into an
> |"    intermediate file format.
>
> Of course they do; they limit image data to have the arbitrarily limits of the
> resulting data-type.

Not true.  Our internal data types do not constrain the type or quantity of
data read in from an input file or written out to an output file.  You are
free to read more about these data structures by checking out the "imintro"
and "tagintro" manual pages in the Image Tools, and libsdsc releases,
respectively.  Alternatively, you can read any of the published papers on
the tools.


> I am sorry; PBMPlus does not need to support these - those packages support
> PBMPlus!

Wavefront, Alias, RenderMan, and so on, do not support PBMPlus.  The majority
of software packages available for workstations, supercomputers, PCs, and
Macs do not support PBMPlus.  This leaves one with the constant need to
convert from one format to another whenever crossing package boundaries.  That
conversion can be done through mutliple tools with PBMPlus, or a single tool
in the SDSC Image Tools.


> By taking an attitude of 'we don't need these low-end formats anyway'
> you first of all are arrogant - let the user decide what he wants if possible,
> not you ! - and second of all, it is easy to write additional converters for
> PBMPLUS (you have done so yourself).

On the contrary, we have not taken the attitude that the low-end formats
don't matter.  They certainly do.  However, we have had to prioritize the
world's vast image file format list to favor those that we and the majority
of our customers need most.  This has resulted in a bias towards formats used
in rendering and scientific visualization.  At some point we fully expect many
of the low-end formats to become part of the Image Tools.

The user is free to implement format readers and writers for the SDSC Image
Tools as well.  The V2.1 release provides access to the file format dispatch
table and provides sample source for the GIF, AVS X, EPS, and PostScript
format handlers.


> |"    PBMPlus image file format support is also strangely sparse.  Some formats
> |"    can only be read, but not written.  Other formats can only be written,
> |"    but not read.
>
> This is due to the nature things get contributed to PBMPLUS: different people
> all over the world have contributed to the package. Some of the stuff is well
> written, others less so. PBMPLUS is not actively supported by people who get
> paid to do so.
>
> |"    The PBMPlus package is not implemented with portability in mind.  There
> |"    are numerous instances within the code where byte order or word size
> |"    is assumed.  Such code will not work on a Cray, for instance.
>
> The PBMPlus package did have portability in mind - the p*m-libraries allow you
> to read and write binary data in a portable way. The people who contributed to
> PBMPLUS may have chosen to not use those. That's unfortunate.
> |"
> |"9.  The SDSC Image Tools were designed to have a consistent and intuitive user
> |"    interface.  All tools have -help arguments to display on-line help
> |"    information distilled from the full man pages.  Argument parsing is
> |"    consistent throughout the package.
>
> Argument parsing can be done consistently in PBMPLUS, the facilities are
> there; again, people chose to use their own when contributing, rather than
> using the provide facilities. It's a matter of attitude: people who use your
> library are not in any way restricted to use the routines you provide, are
> they ?

The hodge-podge nature of the PBMPlus package's support for portability,
consistency of user interface, extent of file format support, and completeness
of documentation are all fully understandable.  This doesn't make them any
less of a problem for the user faced with trying to learn to use the package.


> |"-  PBMPlus is not faster.  It couldn't be.  To do anything in it you first
> |"   have to convert your image file to one of the PBMPlus file formats.  Then,
> |"   after doing your task, you have to convert back out again into the format
> |"   you really want your result in.  At a minimum this requires two extra file
> |"   reads and two extra file writes.
>
> No. Ever heard of pipes ? After all, we are talking high-end machines here!

Indeed.  However, work involved in the conversion to and from the PBM file
formats requires computation time and pipe streams require data storage space
(swap or file system space, it doesn't matter) even when pipes are used
instead of intermediate data files.  Piping of data avoids user-visable temp
files, but does not significantly affect execution times.  Use of piped
commands in PBMPlus is slower than equivalent non-piped tasks done with the
SDSC Image Tools.  This is unavoidable.

The SDSC Image Tools also support piping of data between image file formats.
All tools can read or write all file formats from stdin or stdout, respectively.


> |"   The "ppmquant" is necessary because the rotated image will have more than
> |"   256 colors (an artifact of the rotation algorithm).  Calling "pnmtorast"
> |"   on an image with more than 256 colors generates an RGB RAS file, not an
> |"   8-bit RAS file suitable for display on a SPARCstation.  So, if we want an
> |"   8-bit RAS file, we first have to quantize the image back down to 256 colors.
>
> No, you should use Xloadimage instead.

If one's end goal is to display an image on one's workstation, and one has
Xloadimage, then use of Xloadimage (or any of the great display tools out
there) is certainly the prefered approach.

However, if one's goal is to send the image to someone who might not have
Xloadimage, or to print it to an 8-bit output device, or to display it on a
non-X display, or to process it's color table for color table cycling, or to
use it as an alpha mask in a paint program or rendering system, then use of
Xloadimage is certainly not going to work.  In these cases one must convert the
image back to 8-bit with 256 or fewer colors.  "ppmquant" becomes necessary
and slows down the procedure in PBMPlus.

The example in my posting was to compare two solutions to a generic goal:
rotate an 8-bit image and give back an 8-bit image.  In such a comparison,
the PBMPlus package solution requires more time and hassle than equivalent
actions in the SDSC Image Tools.  I consider this a representative task
for comparison purposes.

You are, of course, free to make your own comparisons in your own real-world
situations.


> Next time, a more objective comparison would be a lot nicer and increase your
> credibility.

You, or anyone else, are certainly encouraged to do so.  Others have.  I
would, however, ask that reviwers of the two packages read both the PBMPlus
and the SDSC Image Tools documentation and papers first.  This will insure
that opinions formed are accurate.


Dave Nadeau                        [e-mail]
Advanced Scientific Visualization Laboratory        [phone]
San Diego Supercomputer Center
P.O. Box 85608   San Diego, CA   92186-9784