ftp.pasteur.org/FAQ//ftp-pasteur-org-FAQ.zip/FAQ/arch-storage/part2

Internet Message Format | 1998-01-17 | 115.5 KB

Path: senator-bedfellow.mit.edu!bloom-beacon.mit.edu!hecate.umd.edu!cs.umd.edu!ra.nrl.navy.mil!whisky.nrl.navy.mil!news.cc.ukans.edu!arclight.uoregon.edu!newsfeed.internetmci.com!199.120.150.5!news.ridgecrest.ca.us!nntp-server.caltech.edu!rdv
Newsgroups: comp.arch.storage,comp.answers,news.answers
Subject: comp.arch.storage FAQ 2/2
Message-ID: <69p5sn$[e-mail]>
From: [e-mail] (Rodney D. Van Meter)
Date: 17 Jan 1998 02:43:35 GMT
Followup-To: comp.arch.storage
Expires: Sun, 1 Mar 1998 00:00:00 GMT
References: <69p5nc$[e-mail]>
Organization: California Institute of Technology, Alumni Association
Keywords: computer mass storage, frequently asked questions, disk,
 tape, RAID, file systems, hierarchical storage management
Approved: [e-mail]
NNTP-Posting-Host: alumni.caltech.edu
Lines: 3037
Xref: senator-bedfellow.mit.edu comp.arch.storage:16639 comp.answers:29699 news.answers:121004

Archive-name: arch-storage/part2
Version: $Header: /nfs/yelo/rdv/comp-arch-storage/faq/RCS/FAQ-2.draft,v 1.37 98/01/16 18:20:09 rdv Exp $
Posting-Frequency: monthly

Rod Van Meter, Joe Stith, and the gang on comp.arch.storage

Information on disk, tape, MO, RAID and SSD can be found in part 1 of
the FAQ.  Part 2 covers file systems, hierarchical storage management,
backup software, robotics, benchmarking, MTBF and miscellaneous
topics.



1.     Standards
1.1.   ANSI X3B5 {None}
1.2.   IEEE Mass Storage System Ref Model (OSSI) {Brief, 6/1/95}
1.3.   ECMA - European Computer Manufacturers Association {None}
1.4.   System Independent Data Format (SIDF)

2.     I/O Related Email Lists

3.     Hierarchical Storage Management
3.1.   Unitree {Brief}
3.1.1. Epoch vs Unitree
3.2.   National Storage Lab {Brief}
3.3.   HIARC {New}
3.4.   Epoch (also known as StorageTek's NearNet) {Brief}
3.5.   Zetaco/NETstor {Brief}
3.6.   R-Squared Infinity IFS 2 {Brief}
3.7.   AMASS
3.8.   Tracer XFS {None}
3.9.   Metior
3.10.  NAStore {Brief}
3.11.  DMF {Brief}
3.12.  FileServ {Brief}
3.13.  Cray Research's Open Storage Manager {Brief}
3.14.  T-mass {None}
3.15.  HP OpenView OmniStorage
3.16.  Platinum NetArchive-HSM {Brief, New}
3.17.  Large Storage Configurations {Brief,New}
3.18.  Unix HSM Vendor List
3.19.  Mainframe
3.20.  PC & PC Server Oriented Packages
3.20.1. HP Optical Jukebox Storage Solution
3.20.2. Chili Pepper Software
3.20.3. Cheyenne ARCserve
3.21.  DATMAN {Brief}
3.22.  Windows NT
3.23.  Other Non-Unix HSM
3.24.  Tapes as Disks {Brief, New}

4.     Backup Software
4.1.   PC-Oriented Backup Packages
4.2.   Unix Packages
4.2.1. Spectra Logic Alexandria
4.2.2. ADSTAR Distributed Storage Manager
4.2.3. NetWorker
4.2.4. BudTool {Brief}
4.2.5. HP OmniBack II {Brief, New}
4.2.6. Workstation Solutions {Brief}
4.2.7. Amanda {Brief, New}
4.2.8. Remote Backup or Mirroring {Brief, New}

5.     Tape and Autochanger Management Software
5.1.   REELlibrarian
5.2.   ANT Medium Changer
5.3.   Tapes 3000 {Brief}
5.4.   Others

6.     Robotics (Autochangers, Jukeboxes, Stackers, Libraries)
6.1.   8mm {Brief}
6.1.1. Exabyte {Brief}
6.1.1.1. EXB-10h
6.1.1.2. EXB-210
6.1.1.3. EXB-220
6.1.1.4. EXB-440/480
6.1.1.5. EXB-10
6.1.1.6. EXB-10i
6.1.1.7. EXB-10e
6.1.1.8. EXB-120
6.1.2. ADIC {Brief, New}
6.1.3. Storage Tek (was Lago) DataWheel {Brief}
6.1.4. ACL {None}
6.1.5. Cambridge On-Line Storage {Brief}
6.1.6. Spectra Logic {Brief}
6.1.7. Qualstar {Brief}
6.2.   3480
6.2.1. StorageTek {Brief}
6.2.2. EMASS (was GRAU) {Brief}
6.2.3. 3590 (Magstar,NTP) {Brief}
6.3.   4mm {Brief}
6.3.1. Cambridge On-Line storage {Brief}
6.3.2. Spectra Logic {Brief}
6.3.3. HP 4mm {Brief}
6.3.4. Storage Tek Datawheel {Brief}
6.3.5. Diverse Logistics Libra {Brief, New}
6.3.6. Qualstar {Brief, New}
6.3.7. ADIC {Brief, New}
6.4.   VHS {Brief}
6.4.1. MountainGate (was Metrum)
6.5.   Digital Linear Tape (DLT) (Quantum) {Brief}
6.5.1. TZ877 {Brief}
6.5.2. TL820 {Brief}
6.5.3. MountainGate
6.5.4. Breece Hill {Brief}
6.5.5. Odetics {Brief}
6.5.6. MediaLogic ADL
6.5.7. ADIC {Brief, New}
6.6.   D-2
6.6.1. Ampex
6.6.2. Odetics
6.7.   ID-1
6.7.1. Sony DMS, PetaSite {Brief}
6.8.   Optical Disk (MO,WORM) Libraries
6.8.1. Hitachi 448 GB optical library
6.8.2. HP MO Autochangers
6.8.3. Maxoptix MO Autochangers
6.8.4. MountainGate {Brief}
6.8.5. DISC DocuStore {Brief}
6.8.6. Kodak {Brief}
6.8.7. Sony {Brief}
6.9.   CD-ROM Jukeboxes
6.9.1. Pioneer
6.9.2. CyberTower {Brief, New}
6.9.3. NSMJukebox {Brief, New}
6.9.4. Nakamichi {Brief, New}
6.9.5. CDI Juke Box Library {Brief,New}
6.9.6. K & S M-200 {Brief, New}
6.9.7. DISC {Brief, New}
6.9.8. Meridian {Brief, New}

7.     File Systems
7.1.   NFS {Brief}
7.1.1. NFS V3
7.2.   AFS {Brief}
7.3.   DFS {Brief}
7.4.   Log based file systems
7.5.   Mainframe File Systems
7.6.   Parallel System File Systems
7.7.   Microsoft Windows NT {Brief}
7.8.   Large Unix File Systems
7.9.   Non-Unix Large File Systems

8.     (Device) Interfaces
8.1.   SCSI {Full}
8.1.1. Single ended vs differential
8.1.2. Asynchronous vs Synchronous Transfers
8.1.3. SCSI-I vs SCSI-II vs SCSI-III
8.1.4. Fast-Wide SCSI
8.1.5. Shared Busses / Performance {Brief}
8.1.6. Cabling/Hot Plugging {Brief}
8.1.7. Third Party Transfers/Separation of Control & Data Paths {Brief}
8.2.   IDE {Brief}
8.3.   IPI {None}
8.4.   HIPPI {Brief}
8.4.1. HIPPI-6400 {Brief}
8.5.   Ultranet {Brief}
8.6.   Ethernet {Brief}
8.7.   FDDI {None}
8.8.   Fibre Channel Standard (FCS)
8.9.   ESCONN/SBCON {Brief}
8.10.  IEEE P1394 (Serial Bus)
8.11.  Serial Storage Architecture (SSA)
8.12.  S2I: IEEE P1285 Scalable Storage Interface
8.13.  Multibus, Unibus, Mainframe Channels, and other history {None}

9.     Other
9.1.   Video vs Datagrade tapes {brief, 5/94}
9.2.   Compression

10.    Benchmarking

11.    Mass Storage Conferences
11.0.1. THIC Tape Head Interface Committee {Brief, New}

12.    MTBF (Mean Time Between Flareups, er, Failures)

13.    Mass Storage Reports

14.    Network-Attached Peripherals {Brief}

15.    Other References
15.1.  Print
15.2.  Web
15.3.  Newsgroups
15.4.  Research Papers

16.    ORIGINAL CALL FOR VOTES

17.    Original Author's Disclaimer and Affiliation:

18.    Copyright Notice

19.    Additional Topics to be added
------------------------------
Subject: [1] Standards
From: Standards

There's a killer supply of computer-related standards at
http://www.cmpcmm.com/cc/. Fibre Channel and several
mass-storage-related items can be found there.

The ANSI and IEEE standards can be purchased in hardcopy form (the
only way some of them are available) from Global Engineering
Documents, [phone] or [phone].

------------------------------
Subject: [1.1] ANSI X3B5 {None}
From: Standards

------------------------------
Subject: [1.2] IEEE Mass Storage System Ref Model (OSSI) {Brief, 6/1/95}
From: Standards

The Storage Systems Standards Working Group now has a WWW page at
http://www.arl.mil/IEEE/ssswg.html.

Version 5 of the model is available via
ftp://swedishchef.lerc.nasa.gov:mass_store/ as the files
ossiv5.ps{1,2,3}.

The OSSI (Open Storage Systems Interconnection) Reference Model (its
new name) "provides the framework for a series of standards for
application and user interfaces to open storage systems." One of its
prime purposes is to define a common vocabulary. Claiming compliance
with the model at the moment has little practical value as far as
interoperation of different pieces from different vendors goes (which
is one of the ultimate aims of the still-distant standards that may
develop from this model).

------------------------------
Subject: [1.3] ECMA - European Computer Manufacturers Association {None}
From: Standards

------------------------------
Subject: [1.4] System Independent Data Format (SIDF)
From: Standards

This is a data format for tapes and removable disks, to facilitate
interchange between hardware and software platforms. See the FAQ at
http://www.mcs.net/~jgast/sidf.html.

------------------------------
Subject: [2] I/O Related Email Lists
From: I/O Related Email Lists

Here is a list of email reflectors for those who need to be deeply
involved in the technical details of various interfaces and standards.



                                                              X3T10/95-010
r0
                                                                April 6,
1995
            I/O Interface Related Reflectors (mailing lists)

              Subscribe/                                    majordomo/
Reflector     Unsubscribe          Broadcast to             listserv
Name          Address              Reflector                keyword
 ------------- -------------------- ------------------------
 -----------------
SCSI          scsi-request@symbios [e-mail]         n/a (human)
              .com
ATA           [e-mail] [e-mail]           ata
ATAPI         [e-mail] [e-mail]         atapi
SSA           [e-mail] [e-mail]           ssa
IDETAPE       [e-mail] [e-mail]       idetape
Disk Attach   [e-mail] [e-mail]   disk_attach
10bit         [e-mail] [e-mail]         10bit
CD-Recordable [e-mail] [e-mail]           cdr
System Issues [e-mail] [e-mail]            si
MultiMedia    [e-mail] [e-mail]           mmc
IEEE P1394    bob.snively@eng.1. [e-mail]            n/a (human)
              com
SFF           bob.snively@eng.1. [e-mail]    n/a (human)
              com
IPI           [e-mail]  [e-mail]        ipi-ext
HIPPI         [e-mail]  [e-mail]      hippi-ext
Fibre Chan.   [e-mail]  fibre-channel-ext@think.
fibre-channel-ext
                                   com
FC IP Prot.   [e-mail]  [e-mail]      fc-ip-ext
PCMCIA        [e-mail]  [e-mail]    pcmcia-gen
FC Class 4    majordomo@northyork. [e-mail]  fc-class4
              hp.com
FC Isoch.     majordomo@northyork. [e-mail].   fc-isoch
              hp.com               com


All of the majordomo and listserv reflectors are automatic.  To
subscribe or unsubscribe, send a message to the subscribe/unsubscribe
address with a line in the message body (not the subject line) of the
following format:

command  reflector_name  [your_email_address]

NOTE: At least for the reflectors at [e-mail], your email
address is optional.  If you include it and it doesn't match the
address in the email headers, there will be a delay while humans
verify your email address.

examples:

subscribe ata
subscribe ssa
subscribe ssa [e-mail]
subscribe atapi
subscribe mmc
subscribe fibre-channel-ext [e-mail]
subscribe pcmcia-gen [e-mail]
unsubscribe ssa [e-mail]
help
lists

The other reflectors are managed by humans who are a little less picky
about the request format, but not quite as prompt.  Please include
your name, email address, phone, and fax numbers in the message body
for the human-managed reflectors.

(with permission from John Lohmeyer, 95/5/10)

------------------------------
Subject: [3] Hierarchical Storage Management
From: Hierarchical Storage Management

HSM systems transparently migrate files from disk to optical disk
and/or magnetic tape, usually robotically accessible.  Then when files are
accessed by a user, they transparently move them back to disk.

Watch for maximum file size limitations, sometimes limited by the
size of the media used, sometimes by the server's OS, and sometimes
neither.

Some offer integrated backup. Some will manage multiple copies of
files for data reliability.

Some offer integrated migration from other systems (ie, file servers
and/or workstations) to the central location disks, then to the central
location robotics. This generally requires changes to the on-disk file
system format on the migration clients.

An item to watch for is that the file management may be exactly like
Unix -- that is, all files appear to be online, and once they're
deleted, they're gone forever, even though the data may still be on
tape.

All of the subsections here are Unix-compatible (in various flavors)
unless indicated otherwise.

Additional Information:

See also _DEC Professional_, February 1993, Page 40 and _Client/Server
Today_, Dec. '94, p. 60.

The System-Managed Storage Guide by Howard W. Miller, $225 for first
copy, $75 for additional copies for same company available from The
Information Technology Institute, 136 Orchard Street, Byfield,
Massachusetts, 01922-1605.

        ([e-mail])

Thomas Woodrow did an evaluation of NAStore, FileServ, DMF and Unitree
in 1993. It can be obtained through
http://www.nas.nasa.gov/NAS/TechReports/RNDreports/RND-93-014/RND-93-014.html
or the Proc. 3rd NASA Goddard Conference on Mass Storage Systems and
Technologies, Oct. 1993, pp. 187--216. Somewhat dated now but
excellent methodology for comparing HSMs.

------------------------------
Subject: [3.1] Unitree {Brief}
From: Hierarchical Storage Management

The uncle of UNIX HSMs. Developed primarily at Lawrence Livermore
National Laboratories.  Commercialized by a company called DISCOS,
then sold to OpenVision.  UniTree was sold to UniTree Software in
December, 1994.  See http://www.unitree.com.

Many versions exist on different hardware platforms, including a
National Storage Lab (NSL) UniTree commercialized by IBM - Fed
Systems. It's also available on SGI, Convex, and Amdahl hardware, at
least.

        See also "Epoch vs Unitree" below

For Convex, try


Jim Wilson
214-497-3085
[e-mail]
Business Development
Data Management Applications
Convex Computer Corporation


For most other platforms, call Open Vision at (800)223-OPEN or
[phone].

New info:


The latest release of UniTree, V1.9.1, has the following changes:
    - Available directly from UniTree Software Inc.
    - Support DEC, HP-UX, SGI, Sun
    - GUI(Tcl/Tk) tools for installation and administration
    - New name database structure
    - Common Message Logger
    - Parallel Migration and Staging
    - Multiple Storage Hierarchy (Optical/Tape)
    - FTP performance improvements (Read/Write 20MBs/16MBs)*
    - NFS performance improvements (Read/Write 3.5MBs/2MBs)*
    - Rule-based dynamic migration
    - Support for new robots (e.g., STK 97xx)
    - Support >2GB disk partitions on Sun
    - 64K File Families
    - Configurable media and drive types
    - Departmental File Server Configuration
    - Compatible with most backup software (Legato, CAM, SMArch)

Demo copy available for download from web site: www.unitree.com

New resellers in Asia, Europe, Australia


* Measured on a dual cpu Sun Ultra3000 with 256MB and 10 disks
--
Francis Kim                Phone: [phone]
Director of Sales and Marketing        FAX:   [phone]
Unitree Software, Inc.            e-mail: [e-mail]
11875 Dublin Blvd. Suite A200E            WWW: http://www.unitree.com
Dublin, Ca. 94568


------------------------------
Subject: [3.1.1] Epoch vs Unitree
From: Hierarchical Storage Management

    (Note: this evaluation is old, and should be taken with a
grain of salt. rdv, 3/96)

        (6/93) We just bought both last year.  We bought an Epoch I
with the 20 GB EO and 327 GB worm.  We will be upgrading it to an
Epoch II soon.  We also bought Unitree from Titan to run on a Silicon
Graphics server and hook up to the STK 3480 silo.  We hope to add more
silos eventually.

Unitree is licensed based on storage capacity while Epoch is not.
There may be an exception to this - STK just began reselling Epoch as
the front end for their silos and I'm not sure how they handle
licensing.

My office mate and I (I handle Epoch, he handles Unitree) have enjoyed
comparing the merits/demerits of each over the last year.  Comparison
in our case is slightly slanted due to the fact that the Epoch has
optical disk while the Unitree system has 3480 tape - so some
observations have more to do with media advantages/disadvantages.


        Unitree

+  Allows large files - can span volumes
+  Allows you to enlarge the cache easily, allows very large
cache
+- Unitree has replaced several UNIX utilities with their own
(FTP, NFS and the file system).  This allows certain features to
work but is generally slower and disallows access to the archive when
you are on the server itself - unless you NFS mount!
+  Allows deleted files to be saved for a specified time
+  Allows multiple copies of files to be kept
+  Data is copied to archive soon after creation
+  Unitree runs on several different platforms
-  Does not allow access to data until it is completely
reloaded
-  Behaves poorly with small files (due to necessary overhead)
-  Unitree is licensed to several vendors, so versions differ
-  NFS access is so slow that we recommend it not be used for
file transfer - only for ls and du, etc.  Use FTP.
-  The Silicon Graphics version is still new and has some
problems

        Epoch

+  Allows access to the data as soon as part of it is loaded
+  Company seems serious about reputation and support
+  The Epoch II is based on a SUN system, with few
modifications
+  Data is copied to archive only when the cache space is
needed
+  All native UNIX transfer methods work - NFS, FTP and RCP
+  Add on products greatly simplify backup and extend
archiving features to other systems.
-  Deleted files are gone forever
-  Currently only available on SUN.  This will change.
-  Cannot span volumes yet - limiting file size
-  Has the SUN limitation of 2 Gb per filesystem.  This would
be a bigger problem if you used it for a 3480 silo.
{Note 2GB of Magnetic Disk limit, not the entire HSM store}
-  Behaves poorly with small files (due to necessary overhead)
-  Since inodes are kept on magnetic cache, you must take
into account the maximum number of files you will ever need.
-  Since inodes are always on disk, certain disk operations
can take forever since all inodes must be examined.
-  Enlarging a magnetic disk filesystem which has associated
archive media requires you to offload all data and then reload it.
If anyone has found another way, I would like to hear about
it.
                   {Others did offer some easier work-arounds}


In all fairness to Titan, they have been addressing any problems and
it has been improving.  Epoch too plans to address some of their
shortcomings.  We are looking forward to growing with both products.

The likelihood that the various flavors of Unitree will standardize
depends on what happens with Discos.  My guess is that some
features/enhancements will be filtered back to the base product
released by Discos.  Bye...

        ([e-mail], 152.61.192.66, Tom Bodoh, USGS/EROS Data
Center, Sioux Falls, SD)

------------------------------
Subject: [3.2] National Storage Lab {Brief}
From: Hierarchical Storage Management

NSL is an industry consortium (American companies only) that has a
version of Unitree, and is creating their own new High Performance
Storage System.

HPSS, among other features, supports striping of removable media, and
full 64-bit files. Some of the work is being done at LLNL, where
UniTree was originally developed.
There's a good overview reachable at
http://www.ccs.ornl.gov/HPSS/HPSS.html.

(rdv,95/1/12)

------------------------------
Subject: [3.3] HIARC {New}
From: Hierarchical Storage Management

HIARC HSM runs on Solaris 2.4 and above.  Slides in at the vnode
layer.  Supports 4mm, 8mm, 3480, DLT, VHS, D-1 and D-2 tape drives,
and appropriate robotics (I don't have a specific list).  Removable
media formats are standard (_which_ standard, I don't know).  Pricing
from $4k to $25k is reasonable for the functionality.  See
http://www.hiarc.com. (rdv, 97/3/20)

------------------------------
Subject: [3.4] Epoch (also known as StorageTek's NearNet) {Brief}
From: Hierarchical Storage Management
        See also "Epoch vs Unitree" in Appendix

------------------------------
Subject: [3.5] Zetaco/NETstor {Brief}
From: Hierarchical Storage Management

NETstor can be reached at [e-mail]

NETstor, Inc. (formerly Zetaco, Inc.) is a leading provider of
hierarchical online mass-storage systems for open systems.  Primarily
NFS accessable systems with magnetic disks and optical-disk libraries.
They have marketing agreements with Digital Equipment Corp, and
Hewlett-Packard.

        ([e-mail])

Netstor was bought by Cheyenne, and is now sold by them
([e-mail], 10/95).

------------------------------
Subject: [3.6] R-Squared Infinity IFS 2 {Brief}
From: Hierarchical Storage Management

Contact: Steve Wine, Manager, Mass Storage Products, R-Squared, 11211
East Arapahoe Rd, Englewood, CO  80112, 303/799-9292 or FAX [phone]

------------------------------
Subject: [3.7] AMASS
From: Hierarchical Storage Management

From Advanced Archival Products. Supports a huge range of devices,
autochangers, and operating systems. Block-based movement of data
between the hard disk cache and tape or optical tertiary storage.
Systems run from a few gigabytes up to at least 12 TB, with prices
dependent on capacity. New versions allow multiple cache disks. Slips
right in to the VFS layer and looks like a normal Unix file system,
with the plusses and minuses that entails. No file versioning or
multiple copies yet. File creation is an Achilles' heel on
performance. Since it's block based, files can be larger than a piece
of media. Separate product DataMgr will migrate files from client
machines to the AMASS server automatically (with FS changes, of
course).

AMASS is now owned by EMASS, and you can find info at
http://www.emass.com/Products/Software_Products/AMASS/AMASS_Top.html.
(rdv, 1996/3/27)

------------------------------
Subject: [3.8] Tracer XFS {None}
From: Hierarchical Storage Management

------------------------------
Subject: [3.9] Metior
From: Hierarchical Storage Management

Metior (pronounced like meteor) is targetting an incredibly broad
market, from laptops with removable media through supercomputers, with
prices from $650(!) to $118K. They handle multiple coordinated copies,
so off-site backup can be automatic. Can do migration for client
machines (with appropriate software licenses and changes to the file
system). The hierarchy seems to be extremely flexible, variable on a
per-user or per-group basis. Machines without client licenses can
mount the Metior FS using NFS. Runs on Suns, SGI, and HP 9000/700. ANT
is new, and they've only got a handful of customers so far, but it
looks _very_ interesting.

(info from [e-mail], written by rdv, so it's my fault if it's
not accurate) (rdv,94/7/7)

More information available on the WWW FAQ version.
Also see them at http://anthill.com.

Automated Network Technologies
3333 South Bannock Street, Suite 945
Englewood, CO 80110   USA
Phone [phone]
FAX [phone]
Email [e-mail]


------------------------------
Subject: [3.10] NAStore {Brief}
From: Hierarchical Storage Management

NAStore is a Unix migrating file system developed by the Numerical
Aerodynamic Simulation program at NASA Ames. It is available through
NASA's software distribution agency, COSMIC. It currently runs only on
Convex with 34x0 cartridges and Storage Tek robots. Looks like a local
file system to users of the Convex.  Available with source.

Info on NAStore can be found on the web at
http://chuck.nas.nasa.gov/NAStore/NAStoreQR.html

        COSMIC's address is :
        University of Georgia
        382 East Broad Street
        Athens, Georgia, 30602-4272, US
        011-706-542-3265
        [e-mail]

For more information on NAStore, contact John Lekashman, [e-mail].
(info from Bill Ross, [e-mail], 94/9/15)

------------------------------
Subject: [3.11] DMF {Brief}
From: Hierarchical Storage Management

Cray Research's Data Migration Facility. The grandaddy of Unix HSM
systems. You can find info on DMF at
http://www.cray.com/product-info/sw/SM/DMF_flyer.html, or call +1 612
683 3897 or email [e-mail]. It's reportedly
running on more than 200 systems, and development is continuing. Large
users are in the hundreds of TB, with millions of files and >1TB/day
through DMF.

Information from:
"Storage Management at Cray Research, Inc", Metcalfe, D.J. and Thompson. D.
"Data Migration Facility Development Update", Lazatella, T.W. and Bannister, N.
Cray User Group, Barcelona, 1996, in press.

([e-mail], 1996/4/2)

------------------------------
Subject: [3.12] FileServ {Brief}
From: Hierarchical Storage Management

From E-Systems. Works with the E-Systems ER-90 (D-2) tape drive and
Odetics robots, as well as 3480 with the Storage Tek ACS 4400. Runs on
Convexes (only?). Supports multiple copies of files. Retrieves only
necessary info from tape to disk before completing request.

Reportedly no longer available on Convex, in beta test on SGI
([e-mail], 10/95)

Now owned by EMASS, info at
http://www.emass.com/Products/Software_Products/FileServ/FileServ_Top.html.

------------------------------
Subject: [3.13] Cray Research's Open Storage Manager {Brief}
From: Hierarchical Storage Management

They have some agreement with Legent Corporation. OSM runs on Sparc
machines, including the Cray Superservers. Price ranges from $500 to
$5,000, which is very cheap for HSM. However, it might only be capable
of migrating among disks -- I don't see any mention of autochangers.
(rdv, 94/12/9)

------------------------------
Subject: [3.14] T-mass {None}
From: Hierarchical Storage Management

------------------------------
Subject: [3.15] HP OpenView OmniStorage
From: Hierarchical Storage Management

Supports multiple types of tertiary media (optical, tape) though it
seems to come originally from their work for their own MO jukeboxes.
Supports multiple types of clients. (info from Herbert Volk
<[e-mail]>, 1995/9/28)

More info available at http://www.hp.com/go/openview.  Now a very broad
storage management suite, covering lots of functionality for
management.  Supports MO, DLT and 8mm as media, though only a limited
number of autochangers. (rdv, 98/1/16)

------------------------------
Subject: [3.16] Platinum NetArchive-HSM {Brief, New}
From: Hierarchical Storage Management

Used to be ASC (Advanced Systems Concept) before being bought by
Platinum. Runs on SunOS, HP, and Domain/OS. Supports numerous optical
jukeboxes. See http://www.platinum.com. (rdv, 96/4)

PLATINUM technology, inc.
1815 South Meyers Road
Oakbrook Terrace, IL 60181
1-800-442-6861  -or-  708/620-5000
e-mail: [e-mail]

------------------------------
Subject: [3.17] Large Storage Configurations {Brief,New}
From: Hierarchical Storage Management

http://www.lsci.com describes their Solaris-based HSM product. Only one
computing platform, but a reasonably broad range of mid- to high-end
peripherals and robotics supported, from little Exabyte autochangers
to the IBM 3494 and STK silos. (rdv, 96/7/23)

------------------------------
Subject: [3.18] Unix HSM Vendor List
From: Hierarchical Storage Management

This list is adapted from _Client/Server Today_, Dec. '94, with some
of my own additions. All the phone numbers are USA (apologies to
international readers for the 800 numbers, but they're all I've got).
I don't know anything about some of these companies; I suspect some of
them work with HSM from other vendors rather than their own packages.

I've indicated on the list various reports of companies OEMing from
each other; this is not out of disrespect for the work involved in
OEMing/supporting or porting such complex software, but an attempt to
divide the HSM vendors into "families" with similar capabilities
(occassionally on very disparate platforms).

Vendor                    Product        Contact
------                    -------        -------
Advanced Archival Products        AMASS        [phone] *
Advanced Software Concepts (ASC)            [phone]
Alphatronix                ASC        [phone]
Artecon                    ASC        [phone]
AT&T CommVault                DataMigrator    [phone]
Automated Network Technologies (ANT)    Metior        [phone] *
Computer Associates International            [phone]
Computer Upgrade                    [phone]
Convex Computer                UniTree        [phone] *
COSMIC (NAStore)                    [phone] +
Cray Research                DMF        (800)BUY-CRAY *
Digital Equipment (DEC)            NETstor        [phone]
Dorotech                        [phone]
Epoch Systems                        [phone] *
E-Systems                FileServ              ?*
File Tek                Storage Machine    [phone]
Fujitsu Computer Products of America    OSM        [phone]
Hewlett-Packard            OmniStorage* ,NETstor    [phone]x8509
HIARC                            [phone]
IBM                    UniTree        [phone]
Introl                            [phone]
Large Software Configurations (LSC)            [phone] *
Legent                    $OSM        [phone]
National Storage Lab (NSL)         HPSS                  +*
NETstor (Cheyenne)            $NETstor    [phone]
(OpenVision                UniTree        [phone] *)
Platinum                NetArchive HSM  [phone] *
Qstar Technologies                    [phone]
Raxco                            [phone]
Software Partners/32                    [phone]
Storage Technology (STK, StorageTek)            [phone]
T-mass                                           ?
Tracer XFS                                  ?
UniTree Software            UniTree        [phone] *

* = Info elsewhere in FAQ
+ = not commercial product
? = no contact info
$ = original developer (no mark indicates OEM)

------------------------------
Subject: [3.19] Mainframe
From: Hierarchical Storage Management

IBM also has HSM for MVS, called, imaginatively, HSM.

There is the storage home page.  http://www.storage.ibm.com/storage/ I
have also found references to System Managed Storage SMS and HSM and
DFHSM (Data Facility Hierarchical Storage Manager) but could find no
online information.  There are probably manuals like DFHSM Version 2
Release 5.0, General Information manual (GH35-0092) if you are a real
glutton for punishment and have a friend at ibm.

So we have ADSM and DFHSM and DFSMS and probably others.  But not much
online information.  Sorry.

A little searching from the http://www.ibm.com might turn up something
too.

(Del Cecchi, <[e-mail]>, 1996/3/27)

------------------------------
Subject: [3.20] PC & PC Server Oriented Packages
From: Hierarchical Storage Management

------------------------------
Subject: [3.20.1] HP Optical Jukebox Storage Solution
From: Hierarchical Storage Management
        Netware 3.11 based, up to 10.4 Gigabytes, includes model 10LC optical
jukebox which has one drive and 16 disks each with 650 MB formatted capacity.
Hewlet-packard (Palo Alto, CA) 800/826-4111.

------------------------------
Subject: [3.20.2] Chili Pepper Software
From: Hierarchical Storage Management

A company from Atlanta, GA named Chili Pepper Software (404-339-1812)
and 3M have gotten together in some fashion to make HSM software for
PCs using QIC. (rdv, 94/9/5)

------------------------------
Subject: [3.20.3] Cheyenne ARCserve
From: Hierarchical Storage Management

Runs on Netware servers. Transparent to most clients, but has a neat
feature: if you use a special TSR and DLL on client PCs, when it has
to retrieve a file from secondary or tertiary storage, it can give you
an estimated retrieval time and the option to abort. [phone],
[phone].
(rdv,95/02/14)

------------------------------
Subject: [3.21] DATMAN {Brief}
From: Hierarchical Storage Management

Simple HSM for 4mm tape drives under MS-DOS. A limited freeware
version is available.

More info at http://www.datman.com.
Voice: 708-369-7112 Fax: [phone] (Kan Yabumoto,
[e-mail], Nov. 1995)

------------------------------
Subject: [3.22] Windows NT
From: Hierarchical Storage Management

Try:

Avail Systems
4760 Walnut St
Boulder, CO  80301
voice: +1.303.444.4018
  fax: [phone]

[e-mail] (Dave Skinner) (95/2/12)

Avail's product, NetSpace HSM, has been selected by Microsoft to be
incorporated into future versions of NT, and also provides a link
between NetWare and IBM's ADSM. NetSpace also runs on Novell NetWare
systems. See http://www.avail.com ("Wight, Risa" <[e-mail]>,
95/10/17)

------------------------------
Subject: [3.23] Other Non-Unix HSM
From: Hierarchical Storage Management

DEC's old Tops-20 OS supported offline files, and would generate an
automatic request to the operator to mount a tape when the user
accessed the file. When you listed a directory, it would show you
which files were online and which off.

DEC's OpenVMS has some sort of support for this now. VMS 6.1 supports
"shelved" files.

There is also the product Virtual Branches, from Acorn Software, which
does HSM for MO and CD-ROM for OpenVMS.


Acorn Software, Inc.
267 Cox St.
Hudson, MA  01749
voice:  [phone]
fax:    [phone]
Internet:  [e-mail]


------------------------------
Subject: [3.24] Tapes as Disks {Brief, New}
From: Hierarchical Storage Management

There are several packages around (mostly for PCs) that will let you
use a tape drive like a disk drive.  Of course, it's _very_ slow
unless it uses some disk-based information as well.

See http://www.tapedisk.com for one such product. (rdv, 96/11/4)

------------------------------
Subject: [4] Backup Software
From: Backup Software

Backup software usually provides some form of management of files,
tapes, and autochangers. Retrieval of files is not automatic (as in
true HSM). These are designed to allow you to recover from disk or
file system failures, and to recover files accidentally (or
maliciously) deleted or corrupted. Some work in conjunction with HSM
systems, which are often vulnerable to the latter class of problems.

I've concentrated here on backup software that supports various
autochangers, as this is of more interest to people in this group than
standalone software for backing up one hard disk onto one tape.

------------------------------
Subject: [4.1] PC-Oriented Backup Packages
From: Backup Software

I don't think any of the PC operating systems come with tape support
built in, so you have to have some 3rd party software to work with
tape. This short list is primarily oriented toward PC servers.
It's partly derived from _PC Magazine_, March 29, 1994, pp. 227-272.

Note that there has been an ongoing discussion of the pitfalls of
Windows 95 and third-party backup software; many in particular are
having trouble with long file names.

Arcada Software - Storage Exec.        (NT)
Avail                    (NT)
Cheyenne Software - ArcServe        (Netware)
Conner Storage Systems - Backup Exec    (Netware)
Emerald Systems - Xpress Librarian
Fortunet - NSure NLM/AllNet
Hewlett Packard - OmniBack II        (NT)
Legato - NetWorker            (Netware)
Mountain Network Solutions - FileSafe
NovaStor                (Netware)
Palindrome - Network Archivist        (Netware, OS/2, Windows)
Palindrome -Backup Director
Performance Technology - PowerSave    (Netware)
Systems Enhancement - Total Network Recall

Arcada is at 800/327-2232 and at http://www.arcada.com.

{Under Construction}(SHMO)

------------------------------
Subject: [4.2] Unix Packages
From: Backup Software

Some people claim "Unix tape support is an oxymoron," so there's a big
market in outdoing tar, dump, dd and cpio.

APUnix - FarTool
Cheyenne - ArcServe (see under PCs, above)
Dallastone - D-Tools
Delta MicroSystems (PDC) - BudTool
Epoch Systems - Enterprise Backup
IBM - ADSM (Adstar Distributed Storage Manager)
Hewlett Packard - OmniBack II
Legato - Networker
Network Imaging Systems
Open Vision - AXXion Netbackup 2.0 Software http://www.ov.com/product/nb.html
Software Moguls - SM-arch
Spectra Logic - Alexandria
Workstation Solutions

{Under Construction}(SHMO)

------------------------------
Subject: [4.2.1] Spectra Logic Alexandria
From: Backup Software

Spectra Logic makes 4mm & 8mm autochangers, but this software supports
other autochangers as well. Has a nice feature that it claims to be
capable of backing up live Oracle, Informix and Sybase databases.
email [e-mail]. (rdv,95/2/14) On the web at
http://www.spectralogic.com

------------------------------
Subject: [4.2.2] ADSTAR Distributed Storage Manager
From: Backup Software

Runs on everything from OS/2, AIX and OS/400 to VSE/ESA, MVS and VM
providing backups for virtually everything you can think of in PCs and
workstations. (800)IBM-3333 or anonymous ftp to index.storsys.ibm.com.
(rdv,95/2/14) http://www.storage.ibm.com/storage/software/software.htm
or http://www.storage.ibm.com/storage/hardsoft/software/html/adsmhome.htm.

------------------------------
Subject: [4.2.3] NetWorker
From: Backup Software

Backup software. See http://www.legato.com. Runs on a wide variety of
platforms and supports a bunch of types of autochangers.

Legato Systems, Inc.
3145 Porter Drive
Palo Alto, CA 94304
Phone: [phone]
Fax Number: 415-812-6032
Fax-on-demand: 415-812-6156

------------------------------
Subject: [4.2.4] BudTool {Brief}
From: Backup Software

PDC Engineering
111 Lindbergh Avenue
Suite C
Livermore, CA 94550 USA
[phone]
FAX [phone]

See http://www.pdc.com.

------------------------------
Subject: [4.2.5] HP OmniBack II {Brief, New}
From: Backup Software

HP's OmniBack II runs on several different platforms, and splits the
functionality up.  The Backup manager appears to run only on NT, but
it can use devices attached to various flavors of Unix, and backs up
ten different kinds of Unix and PC clients.  Now marketed jointly with
OmniStorage, their HSM system, in a (sales) program they call
OpenView.  See http://www.hp.com/go/openview. (rdv, 98/1/16)

------------------------------
Subject: [4.2.6] Workstation Solutions {Brief}
From: Backup Software

See http://www.worksta.com. Runs on a variety of Unix platforms, and
supports a reasonably broad range (20GB-5TB) of autochangers and tape
systems (4mm, 8mm, DLT, VHS). (rdv, 96/7/8)

------------------------------
Subject: [4.2.7] Amanda {Brief, New}
From: Backup Software

Subscribe to [e-mail] and
[e-mail] for some time.  The "current"
distibution of Amanda seems to be from ftp.gps.caltech.edu:/pub/amanda,
with version 2.3.0.3.  A very good backup system, with no dollar
investment.  (David Olsen, <[e-mail]>, 1/23/97)

You'll also find a FAQ on it at
http://ugrad-www.cs.colorado.edu/~teich/amanda.

------------------------------
Subject: [4.2.8] Remote Backup or Mirroring {Brief, New}
From: Backup Software

It's now possible, in several fashions, to backup systems over a
network or even a modem, for recovery from fires and even disk
crashes.

Channel extenders, such as the CHANNELink
http://www.cnt.com/products/clnk/clnk2.htm from CNT and the Symmetrix
Remote Data Facility http://www.emc.com/symmdoc.htm, are used by some
mainframe systems to create remote copies of disks (remote mirroring)
as a disaster recovery measure. Early systems used dedicated fibre or
telephone lines and ran proprietary communications protocols. Newer
systems from CNT are capable of communicating over general-purpose
wide-area networks, thus saving the costs of the dedicated lines.

It's also possible to backup PCs over your modem in an incremental
fashion, through your ISP; one example is http://www.telebackup.com.

Two other companies that do this over the Internet (out of, I believe,
more than 30) are Connected Corp., Framingham, MA; Virtual Technology
Corp., Minneapolis, MN.


------------------------------
Subject: [5] Tape and Autochanger Management Software
From: Tape and Autochanger Management Software

This category of software can overlap with both HSM and backup, above,
and basic tools are often available from the autochanger hardware
vendors, below.  New additions to this category welcome -- I'm sure
there are numerous vendors I don't know. Functionality varies widely,
from rudimentary "move the cartridge" interfaces to sophisticated
tape-tracking databases. (rdv, 95/6/1)

------------------------------
Subject: [5.1] REELlibrarian
From: Tape and Autochanger Management Software

Actually a whole set of software tools from Storage Tek, available
through Software Clearing House, http://www.sch.com/stor001.html.
Manages different types of media for you, including 3480 in STK silos,
under MVS or Unix. (rdv from [e-mail], 95/6/1)

------------------------------
Subject: [5.2] ANT Medium Changer
From: Tape and Autochanger Management Software

There is a public version of a Solaris 2.x Medium Changer driver
with a set of command line utilities in our FTP server.
Only restriction is that you cannot bundle it with another product
or resell it (intended for end-user use only).

anthill.com:/pub/distrib/mc/solaris2.x/medium_changer.tar.Z
or http://anthill.com/techsupport.html (Tim Sesow, ANT, 1995/9/21)

------------------------------
Subject: [5.3] Tapes 3000 {Brief}
From: Tape and Autochanger Management Software

Tapes3000 is a UNISON/TYMLABS product that puts a label on a reel
tape, DDS, any kind of media storage and adds it to a "tapes database"
so you do not have to manually log and label backup tapes or special
request tapes, and possibally make a mistake. You can also use this
for unlabeled media, but then you would have to manually log the
media. You are able to set a "dataset" for differrent retentions
(Generations, weekly, monthly, daily etc). Then when those criteria
are met the program will automatically scratch those tapes, then you
run a report and it will give you a list of what scratched for that
day, week, or whatever time you want to run the job.

Tapes3000 is part of a package that you can receive called MAESTRO
which is a job scheduler program. Not an autochanger control package,
just tape management software. (R Johnson
<[e-mail]>, 1996/3/25)

------------------------------
Subject: [5.4] Others
From: Tape and Autochanger Management Software

Many of the HSM (including EMASS, above, with their VolServ) and
backup vendors also sell simple autochanger control interfaces. Check
with them.

Some things I've read indicate that one or more of the
university-based projects ought to have a freely available autochanger
controller; if anybody has any info on this let me know.

------------------------------
Subject: [6] Robotics (Autochangers, Jukeboxes, Stackers, Libraries)
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

I use the term "robotics" to refer to access to multiple removable
volumes by a fewer number of drives without a person.  This includes
sequential stackers, as well as random access robotics.

A stacker typically is capable of taking (literally) a stack of tapes
and putting them into the drive one at a time, in order. No random
access to specific tapes, as with a full-function autochanger.
Stackers typically are limited to 8-10 cartridges, and are used by
people whose backups have exceeded the size of one cartridge.

In the larger media formats, such as D-1, D-2, Betacam, etc., the
traditional manufacturers of broadcast autochangers, such as Asaca,
Odetics, Sony, etc. have products that are easily adaptable to storage
use.

The August 1996 issue of Byte magazine has an article comparing 12
tape autochangers.  It is a little misleading, not mentioning any of
the truly large library systems, and only one midrange, whose capacity
is quoted assuming DLT 7000 tape drives, which is never mentioned.  In
addition, much of their testing is more related to the drives than the
autochangers.

------------------------------
Subject: [6.1] 8mm {Brief}
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

------------------------------
Subject: [6.1.1] Exabyte {Brief}
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

Phone: 800/EXABYTE, 1685 38th st, Boulder, CO 80301, Fax [phone].
On the web at http://www.exabyte.com.

------------------------------
Subject: [6.1.1.1] EXB-10h
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

Current model, 10 cartridges, one drive.  Not Mammoth compatible.  70
GB, uncompressed.  (rdv,96/8/29).

------------------------------
Subject: [6.1.1.2] EXB-210
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

2 drives, 11 cartridges, not Mammoth compatible.

------------------------------
Subject: [6.1.1.3] EXB-220
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

2 drives, 20 cartridges, Mammoth compatible (rdv,96/8/29).

------------------------------
Subject: [6.1.1.4] EXB-440/480
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

40 or 80 cartridges, 2 or 4 drives, Mammoth compatible. 1.6 TB
uncompressed, with Mammoth.  (rdv,96/8/29).

------------------------------
Subject: [6.1.1.5] EXB-10
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

Ten cartridges, one full-height drive.  Original 10 cartridge robot.
No robotic intelligence, when one tape comes out, the robot mounts the
next tape in sequence (i.e. a kind of stacker).  Button selectable to
loop back to the first tape or to stop at the end.  Discontinued.

------------------------------
Subject: [6.1.1.6] EXB-10i
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

Ten cartridges, one full-height drive. Released
shortly after the EXB-10.  Includes SCSI attachment to robotics.  Now
nearly replaced by the EXB-10e.  Discontinued.

------------------------------
Subject: [6.1.1.7] EXB-10e
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

Ten cartridges, one full-height drive.  Announced around 4/93.
Includes better control panel and display than EXB-10i.  Drive mounted
horizontal and tape magazine at slight angle (rather than vice-versa
in EXB-10i).  Discontinued.

------------------------------
Subject: [6.1.1.8] EXB-120
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

Holds 120 8mm cartridges, up to four full-height drives.
Discontinued.

------------------------------
Subject: [6.1.2] ADIC {Brief, New}
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

I'm not sure if ADIC manufactures or OEMs their robotics, but they
apparently sell to end users.  They have 8mm, 4mm and DLT autochangers
in a variety of small to medium sizes, up to about a terabyte.  See
http://www.adic.com. (rdv,97/3/18)

------------------------------
Subject: [6.1.3] Storage Tek (was Lago) DataWheel {Brief}
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

 Holds 54 8mm tape cartridge in a carousel with 2 8mm drives. The
carousels are removable. Now Storage Tek, used to be a small company
called LAGO, which apparently no longer exists.

You'll find info at: http://www.stortek.com/StorageTek/9708.html. (rdv,
updated 1996/3/22)

------------------------------
Subject: [6.1.4] ACL {None}
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

------------------------------
Subject: [6.1.5] Cambridge On-Line Storage {Brief}
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)
        Sixty and 240 GB libraries, 713/981-3812

------------------------------
Subject: [6.1.6] Spectra Logic {Brief}
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

Spectra Logic makes SCSI-controlled 8mm and 4mm (DAT) autochangers.
One to four drives, with 20 to 60 slots. Capacity currently up to 600
GB of DDS-2 (4mm) or 300 GB 8mm. Early models (STL-6000 & STL-8000)
were a rotating carousel. Newer ones use an arm and the tapes don't
move.

Supported by a variety of software vendors. List prices of $9K
(Spectra 4000/20 slots, one DDS-2 drive) to $31K (60 slots with four
drives and barcode support) including drives.

They also make a thing called TapeFrame, which consists of several of
their autochangers working in conjunction, with capacities up to 2.2
TB.

U.S.: 1-800-833-1132 or 303-449-6400

(Britt Terry, [e-mail], 95/1/12)

See also under backup software, and on the web at
http://www.spectralogic.com.

------------------------------
Subject: [6.1.7] Qualstar {Brief}
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

Makes 8mm libraries that hold 10 to 120 cartridges and 2 to 6 drives.
tel:[phone] fax:[phone] or http://www.qualstar.com
(rdv,95/2/14)

Our TLS-4000 8mm library family now supports the Sony SDX-300C drive.
Production shipments have started and enduser installations have
occurred. Early field reports are completely positive.

TLS-4000 also supports Exabyte Mammoth and 8505XL drives.

(Bob Covey, [e-mail], 96/10/22)

------------------------------
Subject: [6.2] 3480
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

------------------------------
Subject: [6.2.1] StorageTek {Brief}
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

Storage Tek makes huge autochangers, referred to as silos, round and
several (~5) meters in diameter. They hold 6,000 3480-style tapes. At
original 3480 densities, that's only 1.2 TB per silo, but capacities
have gone up to (I think) 800 MB/cartridge, and are poised for a HUGE
jump if Storage Tek gets their Redwood tape drive finished (in beta
test, 12/94), up to 20 GB/cartridge, 120 TB/silo.

There is a smaller silo, known as WolfCreek, that holds 500-1000
cartridges.

STK also OEMs a 3480 autochanger from Odetics. Holds ~260 cartridges,
I think, in rotating drum, with room for ?2? tape drives above it.
(rdv,95/1/12) However, I couldn't find any info about this on the web
site.

They also have a web site at http://www.stortek.com. (95/5/16,rdv)

All but the Odetics (known as Ocean, I think) are Redwood-compatible.

The new 9710 (codenamed Panther) can handle both DLT and 3480
cartridges in a mini-tower. ([e-mail], 1996/3/12)

------------------------------
Subject: [6.2.2] EMASS (was GRAU) {Brief}
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

Grau, a German manufacturer, makes high-end, very large capacity
mixed-media autochangers known as the ABBA series, targetted I believe
primarily at the IBM mainframe market. (rdv,94/11/7)

Bought by EMASS, see http://www.emass.com.  They support 3480, D-2, MO,
VHS, DLT, 8mm all in one robot, so they renamed the autochanger series
the AML, Automated Mixed-Media Libraries.

------------------------------
Subject: [6.2.3] 3590 (Magstar,NTP) {Brief}
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

MountainGate has announced that they will have, later this year, a
300-cartridge autochanger.

IBM of course makes numerous autochangers for NTP; the 3494 and 3495
models both support it. More info at
http://www.storage.ibm.com/storage/hardsoft/tapls.htm. (They probably
have smaller libraries, too.)

Word in the newsgroup has it that STK robots won't support Magstar due
to the rivalry between IBM and STK.

(rdv,1996/3/12)

------------------------------
Subject: [6.3] 4mm {Brief}
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

------------------------------
Subject: [6.3.1] Cambridge On-Line storage {Brief}
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)
        Libraries of 120 and 40 GB, 713/981-3812

------------------------------
Subject: [6.3.2] Spectra Logic {Brief}
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

Spectra Logic makes SCSI-controlled 8mm and 4mm autochangers. See
above under 8mm autochangers.

------------------------------
Subject: [6.3.3] HP 4mm {Brief}
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

I think HP makes their own 4mm autochangers.

------------------------------
Subject: [6.3.4] Storage Tek Datawheel {Brief}
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

The 4mm version. 25 cartridges, so up to 100GB uncompressed. Info at
http://www.stortek.com/StorageTek/9704.html.

------------------------------
Subject: [6.3.5] Diverse Logistics Libra {Brief, New}
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

Two libraries, the Libra-8 and Libra-16, with 8 or 16 slots (32 or 64
GB uncompressed) and one DAT drive.  Info at http://www.dilog.com or
http://www.dlig.ch (Europe), [e-mail] or
[e-mail]. ([e-mail] (Marc SCHAEFER), 96/8/6)

------------------------------
Subject: [6.3.6] Qualstar {Brief, New}
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

See http://www.qualstar.com.

We are now shipping our TLS-2000 4mm tape library family. This product
line consists of 6 models ranging from 1-2 drives with 18 tapes, to 1-4
drives with 144 tapes. All units include a mailbox and barcode support. I
believe that the TLS-24144 is the largest 4mm library in production.
TLS-2000 supports Seagate, Sony and HP DDS-2 drives and we are about to
start testing the Sony SDT-9000 DDS-3 drive.
(Bob Covey, [e-mail], 96/10/22)

------------------------------
Subject: [6.3.7] ADIC {Brief, New}
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

I'm not sure if ADIC manufactures or OEMs their robotics, but they
apparently sell to end users.  They have 8mm, 4mm and DLT autochangers
in a variety of small to medium sizes, up to about a terabyte.  See
http://www.adic.com. (rdv,97/3/18)

------------------------------
Subject: [6.4] VHS {Brief}
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

------------------------------
Subject: [6.4.1] MountainGate (was Metrum)
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

Metrum's data storage division was bought by Lockheed Martin and
renamed MountainGate.

Autochangers for their VHS-based high-capacity (20GB, 2 MB/sec.) tape
drive. They now have a stacker available for standalone drives.

Library of 960 GB (RSS-48b) holds 2 drives and 48 cartridges in a
rotating drum.

Library of 12 TB (RSS-600b) holds 5 drives and 600 cartridges in less
than 20 square feet of floor space. The tapes are held in rotating
drums on each side, with the drives in a rack in between.

OEMs through Convex, IBM, and a host of resellers. Integrated with
various backup and HSM packages, including UniTree from Convex & IBM,
and AMASS from AAP.

See MountainGate also under MO and DLT autochangers.

MountainGate
A Lockheed Martin Company
9393 Gateway Drive
Reno NV
89511-8910
702-851-9393 Phone
702-851-5533 Fax

See them on the web at http://www.mountaingate.com, but as of today
(1996/3/19) doesn't have much on the high-end products.

------------------------------
Subject: [6.5] Digital Linear Tape (DLT) (Quantum) {Brief}
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

T* names are DEC's names, DLT2* names are OEM names.

------------------------------
Subject: [6.5.1] TZ877 {Brief}
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

        One TZ87 tape drive, 7 cartridges, each 10GB native
        Presumed to be the same as the DLT2700 library.
        Ref: Digital's Customer Update, March 14, 1994


------------------------------
Subject: [6.5.2] TL820 {Brief}
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

        Holds 3 TZ87 tape drives, 264 catridges, five libraries attachable
        Presumed to be Odetics made (714/774-5000)
        About $150K U.S.
        Ref: Digital's Customer Update, March 14, 1994

------------------------------
Subject: [6.5.3] MountainGate
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

At Comdex '94 in Vegas, Metrum (now MountainGate) introduced the D-900
(900 cartridges, up to 20 drives, 9TB uncompressed for DLT-2000) and
D-360 (360 cartridges, up to 8 drives, 3.6 TB uncompressed for
DLT-2000) DLT autochangers. There is an expansion unit with 480
cartridges which may hold two drives. Up to eight D-360 or D-480 units
can be connected via passthrough. They also introduced 28 and 60
cartridge DLT autochangers. Customer shipments starting in early '95.

See above under VHS for contact info.

------------------------------
Subject: [6.5.4] Breece Hill {Brief}
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

Breece Hill makes two small (28 and 60 cartridges) DLT autochangers.
On the web at http://www.csnet.net/breece_hill/

Breece Hill Technologies, Inc.
6287 Arapahoe Avenue
Boulder, Colorado 80303 USA

For more Information 1-800-941-0550 or 303-449-2673

------------------------------
Subject: [6.5.5] Odetics {Brief}
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

Odetics makes a series of DLT libraries that hold, in the basic
configuration, 3 DLT drives and 264 cartridges. See
http://www.odetics.com

------------------------------
Subject: [6.5.6] MediaLogic ADL
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

MediaLogic ADL, Inc.
1965 57th Court
Boulder, CO 80301
Voice: 303-939-9780
Fax: [phone]
email: [e-mail]

They have desktop autochangers up to 26 DLT cartridges. See also
http://www.csn.net/adlinfo/ on the web. Also have 4mm and 8mm
autochangers that are similar. I don't know if they manufacturer these
or OEM them. (rdv, 1996/3/19)

------------------------------
Subject: [6.5.7] ADIC {Brief, New}
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

I'm not sure if ADIC manufactures or OEMs their robotics, but they
apparently sell to end users.  They have 8mm, 4mm and DLT autochangers
in a variety of small to medium sizes, up to about a terabyte.  See
http://www.adic.com. (rdv,97/3/18)

------------------------------
Subject: [6.6] D-2
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

------------------------------
Subject: [6.6.1] Ampex
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)
Ampex makes their own autochangers for the DST DD-2 tape drive (see
part 1 of the FAQ).

DST 410 Automated Cartridge Library:
Up to 1.2 terabytes capacity (uncompressed) in 7 square feet of floor space.
All 3 cartridge (cassette) sizes supported - 25, 75, 165 gigabytes
(uncompressed).
SCSI Medium Changer Commands or Ethernet NetSCSI protocol.
Console mounted configuration.
Single unit price:  $150K.

DST 810 Automated Cartridge Library:
Up to 6.4  terabytes (uncompressed) in 21 square feet of floor space.
Robotic performance of 600 cartridge exchanges per hour.
Average access time to any file less than 30 sec. (including cartridge
exchange, drive load and search to data).
1 to 4 tape drives per library.
Ethernet NetSCSI protocol robotics control.
Starting single unit price:  $300K.

([e-mail], 94/12/23)

------------------------------
Subject: [6.6.2] Odetics
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

Odetics makes a thing called a DataTower that holds ~250 S-size D-2
cartridges. It used to be, but is no longer, sold through EMASS for
use with the ER-90 (the Ampex/EMASS D-2 drive). It's a small silo that
sits in front of one rack of drives.

They also make an expandable library known as the DataLibrary, with a
maximum capacity of ten petabytes(!) (ten million gigabytes). A robot
handler runs on a track down an aisle lined with cartridges, and tape
drives at one (both?) end(s) of the aisle. I think the aisles can vary
in length, and they can be lined up next to each other and I believe
cartridges will pass between them.

(Note: since their acquisition of GRAU (above) EMASS no longer sells
Odetics. I don't know if these are still available directly from
Odetics and who you'd get to do the integration work. (rdv, from
[e-mail] (Dave Barnes), 1996/3/22))

------------------------------
Subject: [6.7] ID-1
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

------------------------------
Subject: [6.7.1] Sony DMS, PetaSite {Brief}
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

Sony sells three autochangers for their ID-1 line of tape drives,
based on their broadcast line of autochangers. These are known as the
DMS Series, models 24, 300M, and 700M. Not surprisingly, they hold,
respectively, 24 (S,M, or L cassettes), 300 (M only) and 700 (M only)
cassettes for capacities of 2.3, 13, and 30 terabytes.

They have also announced something called PetaSite, which they claim
expands to 3 petabytes and supports both ID-1 and DTF in a single
system.

(rdv, 1996/3/22)

------------------------------
Subject: [6.8] Optical Disk (MO,WORM) Libraries
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

Several other Japanese manufacturers make optical libraries, I think,
mostly in support of their own drives. (SHMO)

------------------------------
Subject: [6.8.1] Hitachi 448 GB optical library
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)
12-inch worm, up to 7GB per platter, 2-4 drives, additional cartridge
expansion unit increases capacity 560 GB to 1,008 GB.
Drive rates up to 2.22 MB/sec.
Phone: 800/HITACHI

------------------------------
Subject: [6.8.2] HP MO Autochangers
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

Makes several models, from 16 disks and one drive up to 144 disks and
?4? drives. These are very popular.

------------------------------
Subject: [6.8.3] Maxoptix MO Autochangers
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

Makes several models in the MaxLyb series, the 52, 120 and 180, which
correspond to the capacity in gigabytes for 1.3 GB drives. They hold,
respectively, 2 (52), 2 or 4 (120) and 2, 4 or 6 (180) drives.

They also have a fairly mysterious thing called the Axxis^26, a "high
speed network file retrieval & backup server," which is obviously an
MO autochanger, apparently bundled with a license for Palindrome
Backup Director, suitable for attaching to your Netware file server?

tel: [phone]
fax: [phone]

(rdv,95/02/14)

------------------------------
Subject: [6.8.4] MountainGate {Brief}
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

Now has the OSS-626, which holds 450-626 disks and 2-24 full-height HP
drives. Also a new expandable multi-chassis autochanger similar to the
D-360 DLT autochanger is available.

See above under VHS for contact info.

------------------------------
Subject: [6.8.5] DISC DocuStore {Brief}
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

Makes large libraries (up to ~1,000 5.25" MO catridges, 2.6TB for
standard MO or 4.6TB for non-standard); see
http://www.discjuke.com. (Stephen Fister <[e-mail]>, 96/8/7)

------------------------------
Subject: [6.8.6] Kodak {Brief}
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

Kodak makes their own autochangers for their large (?12"?) optical
drive.

------------------------------
Subject: [6.8.7] Sony {Brief}
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

Sony makes their own jukeboxes for their 12" WORMs and for 5.25" MO.
http://www.sel.sony.com/SEL/ccpg/storage/scontent.html is the place to
start.

------------------------------
Subject: [6.9] CD-ROM Jukeboxes
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

------------------------------
Subject: [6.9.1] Pioneer
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

From: [e-mail] (Mike Caplinger)
Subject: driver software for Pioneer DRM-5004X CDROM jukebox
Date: Tue Aug 23 10:09:00 PDT 1994
Organization: UTexas Mail-to-News Gateway
Lines: 13

Pioneer recently announced their DRM-5004X CDROM jukebox, which has
four quad-speed drives and holds 500 CDs for under $20,000.

        Mike Caplinger
        [e-mail]

Pioneer also has a 6-disk mini-changer, where SCSI LUNs 0-5 correspond
to the individual disks; accessing one causes a mount. (Brian A Berg
<[e-mail]>, 1996/3/29)

There's also an 18-disk model. You can find info on all three at
http://www.pgb.pioneer.co.uk (rdv, 96/8/5)

------------------------------
Subject: [6.9.2] CyberTower {Brief, New}
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

http://www.cyberdatasys.com has frustratingly little info on a product
that apparently is 7 CD-ROM drives made to behave like a single SCSI
target. Not really an autochanger, more of an array. Not sure who the
manufacturer is; the same unit is available from Procom
http://www.procom.com. (rdv,96/8/5)

------------------------------
Subject: [6.9.3] NSMJukebox {Brief, New}
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

http://www.nsmjukebox.com describes what they call "the universe's
fastest CD-ROM jukebox". 150 platters (90GB), up to four drives. (rdv,
96/8/5)

------------------------------
Subject: [6.9.4] Nakamichi {Brief, New}
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

A 4-disk changer built on into an 8x
reader. http://www.nakamichicdrom.com. (rdv,96/8/5)

------------------------------
Subject: [6.9.5] CDI Juke Box Library {Brief,New}
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

A 28-disk changer (standalone network server?) with up to four drives,
and a built-in PC w/ 128 MB RAM and a 1GB disk. Available from
http://www.cdstorage.com. (rdv,96/8/5)

------------------------------
Subject: [6.9.6] K & S M-200 {Brief, New}
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

A 200-disk autochanger. Available from
http://www.cdstorage.com. (rdv,96/8/5)

------------------------------
Subject: [6.9.7] DISC {Brief, New}
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

Makes large libraries (up to ~1,500 media slots and up to 32 drives);
see http://www.discjuke.com. (Stephen Fister <[e-mail]>,
96/8/7)

------------------------------
Subject: [6.9.8] Meridian {Brief, New}
From: Robotics (Autochangers, Jukeboxes, Stackers, Libraries)

CD Net Universal Server from Meridian, http://www.meridian-data.com.
Not really an autochanger, but an array of CD-ROM drives in a box with
an NFS or Netware interface.  (rdv, 97/6/30)

------------------------------
Subject: [7] File Systems
From: File Systems

This topic is also discussed frequently in comp.os.research.
See http://www.maths.tcd.ie/scrg/os-faq.html.

------------------------------
Subject: [7.1] NFS {Brief}
From: File Systems

The Network File System, originally developed by Sun Microsystems and
now pretty standard in the Unix world, and clients exist for PC, Mac,
VMS, and other non-Unix OSes. V2, the common version, supports single
files only up to 2^32 (4GB) bytes. I'm not sure if there are any
limits to a file system size under NFS, other than those imposed by
the client and server OSes (SHMO).

NFS is defined in RFC 1094. V3 is now RFC 1813.

There is at least one newsgroup devoted specifically to NFS:
comp.protocols.nfs.

------------------------------
Subject: [7.1.1] NFS V3
From: File Systems

NFS V3 supports 64-bit files and write caching.

The first implementation was from Digital with DEC OSF/1 V3.0 for
Alpha AXP. Silicon Graphics supports it on IRIX 5.3. Cray will support
it on UNICOS 9. I don't know about other vendors but I have heard
rumours that the releases coming in the second half of 1995 will
support it.

Further information on NFS V3 can be found from
gatekeeper.dec.com:pub/standards/nfs/NFS3.spec.ps.Z

([e-mail], 95/1/22)

Solaris 2.5, available Nov. 95, is reported to have V3 support.
Network Appliances have it as of 3.0, Sept. 95. ([e-mail] (Guy
Harris), 95/10/6)

------------------------------
Subject: [7.2] AFS {Brief}
From: File Systems

The Andrew File System (SHMO). Allows naming of files worldwide as if
they were a locally-mounted FS (from cooperating clients, of course).

There's an "alt" group for AFS - "alt.filesystems.afs". Available
commercially from Transarc.

------------------------------
Subject: [7.3] DFS {Brief}
From: File Systems

Another remote file system protocol that supports large files. I don't
know anything about it, or if any implementations really exist yet.

------------------------------
Subject: [7.4] Log based file systems
From: File Systems

Further Information:
    %z InProceedings
    %K hpdb:Rosenblum91
    %s [e-mail] (Thu Oct 17 11:12:07 1991)
    %A Mendel Rosenblum
    %A John K. Ousterhout
    %y UCBCS.
    %T The design and implementation of a log-structured file system
    %C Proc. 13th SOSP.
    %c Asilomar, Pacific Grove, CA
    %p ACM. SIGOPS
    %D 13 Oct. 1991
    %P 1 15
    %x This paper presents a new technique for disk storage management
    %x called a log-structured file system.  A log-structured file system
    %x writes all modifications to disk sequentially in a log-like
    %x structure, thereby speeding up both file writing and crash
    %x recovery.  The log is the only structure on disk; it contains
    %x indexing information so that files can be read back from the log
    %x efficiently.  In order to maintain large free areas on disk for
    %x fast writing, we divide the log into segments and use a segment
    %x cleaner to compress the live information from heavily fragmented
    %x segments.  We present a series of simulations that demonstrate the
    %x efficiency of a simple cleaning policy based on cost and benefit.
    %x We have implemented a prototype log-structured file system called
    %x Sprite LFS; it outperforms current Unix file systems by an order of
    %x magnitude for small-file writes while matching or exceeding Unix
    %x performance for reads and large writes.  Even when the overhead for
    %x cleaning is included, Sprite LFS can use 70% of the disk bandwidth
    %x for writing, whereas Unix file systems typically can use only
    %x 5--10%.

([e-mail])

Also, these papers:

Ousterhout and Douglis, "Beating the I/O Bottleneck: A Case for Log-
structured File Systems", Operating Systems Review, No. 1, Vol. 23, pp.
11-27, 1989, also available as Technical Report UCB/CSD 88/467.

Rosenblum and Ousterhout, "The Design and Implementation of a Log-
Structured File System", ACM SIGOPS Operating Systems Review, No. 5, Vol.
25, 1991.

Seltzer, "File System Performance and Transaction Support", PhD Thesis,
University of California, Berkeley, 1992, also available as Technical
Report UCB/ERL M92.

Seltzer, Bostic, McKusick and Staelin, "An Implementation of a Log-
Structured File System for UNIX", Proc. of the Winter 1993 USENIX Conf.,
pp. 315-331, 1993.

listed from the man page for mount_lfs under FreeBSD-2.1.5. (rdv, 97/1/17)

------------------------------
Subject: [7.5] Mainframe File Systems
From: File Systems

The WWW FAQ contains some information about mainframe file systems.

------------------------------
Subject: [7.6] Parallel System File Systems
From: File Systems

This discussion comes up occassionally on comp.arch and
comp.os.research. I don't know which newsgroups/mailing lists the PIO
(Parallel I/O) people hang out in, but it doesn't seem to be here.
They show up occassionally in comp.sys.super and comp.parallel.  They
do have their own conferences, though.

The important work seems to be going on with the supercomputing gang
-- LLNL, CMU, Caltech, UIUC, Dartmouth, ORNL, SNL, etc. Work is also
being done by the parallel database community, including vendors such
as Teradata.

A paper presented at the ACM International Supercomputing Conference
in 1993 showed what to me seemed to be pretty appalling performance
for reading data and distributing it to multiple processors on an
Intel Delta supercomputer (sorry I don't have the reference in front
of me). (rdv, 94/8/12) The paper is old, now, and the Intel guys say
they have improved performance to up to 130 MB/sec. on the new Paragon
using their Parallel File System (PFS).

There is an excellent web site on parallel I/O at Dartmouth:
http://www.cs.dartmouth.edu/pario.html

There is also a mailing list housed at Dartmouth,
[e-mail].

The annual conference is I/O in Parallel and Distributed Systems
(IOPADS); 1997's is co-located with Supercomputing '97 in San Jose,
Nov. 17.  Papers are due March 25, 1997.  See
http://www.cs.dartmouth.edu/iopads.

------------------------------
Subject: [7.7] Microsoft Windows NT {Brief}
From: File Systems


I seem to recall that NT supports 64-bit file systems for its own
native file systems? Anybody know for sure (SHMO)? (rdv, 94/8/24)

From *Inside the Windows NT(TM) File System*, by Helen Custer:

"NTFS allocates clusters and uses 64 bits to number them,
which results in a possible 2^64 clusters, each up to 4KB.  Each
file can be of virtually infinite size, that is, 2^64 bytes
long."

"Clusters" can be between 512 and 4K bytes.

The Win32 API supports 64-bit file sizes, albeit in a cheesy fashion
reminiscent of V6 UNIX - no 64-bit integral types used, just pairs of
32-bit integral types. ([e-mail] (Guy Harris), 95/10/6)

------------------------------
Subject: [7.8] Large Unix File Systems
From: File Systems

There is now an industry group working on standardizing an API for
files larger than 2 GB (the max size normally supported on most Unix
systems). More info as I get it. The WWW-enabled can have a look at
http://www.sas.com:80/standards/large.file and see the various
proposals on the table.

Note that it is VERY easy to confuse whether an OS supports _files_
larger than 2 GB or _file systems_ larger than 2 GB. My table lists
some of both (thanks to [e-mail] (Benjamin Z.  Goldsteen),
Ed Hamrick ([e-mail]) and Peter Poorman ([e-mail])
for much of this information).


It is straightforward for systems with 64-bit integers to support
64-bit files; for systems with 32-bit integers it is more complex. On
most 32-bit systems the offsets passed around inside the kernel (most
importantly, at the VFS layer) the file offsets and sizes tend to be
passed as 32-bit (signed) integers, meaning no files >2^31.

On most systems, the argument to lseek is of type off_t, which (on
SunOS and Linux, and plausibly on OSF/1 and others) is declared in a
header file as "typedef long off_t;".

For clients to really have access to large files, three pieces are
required: local FS support, an appropriate network protocol, and
server support for 64-bit FSes. For FTP access, I believe _literally_
inifinitely large files are possible, but I'm not sure(SHMO). For NFS
access, NFS V2 supports only 2GB files. NFS V3, just becoming
available now, supports full 64-bit files, I believe (anybody have a
reference to the docs? RFC? SHMO). With the notable exception of
Unitree (which does not use, depend on, or appear as, a local FS on
the server), server support for 64-bit files is provided only when the
server's own local FSes are 64-bit.

Even for the systems that _do_ support large files, not all are
programmer or user-transparent for supporting large files. UniCOS is,
OSF/1 is, ConvexOS is not (there are two system calls, lseek and
lseek64, with 32-bit and 64-bit file offsets, respectively, though the
Fortran interface is transparent).

This brings up the related issues.  A complete large files implementation
needs not only the system calls, but also the stdio library and the runtime
libraries for the languages (Fortran, Cobol,...).  Further, system utilities
(sed, dd, etcetera) need to be capable of dealing with large files.

(It has been pointed out that the GNU C compiler runs on most of these
machines, so it is possible to use "long long" as a 64-bit int on
them, but what matters for file systems is the system compiler.)

Here's the start of a table on these. Really such a simple table can't
do the problem justice, but it'll give you an idea. Keep in mind that
many of these systems support many file system types; I've listed only
the most interesting so far from this point of view. I'd like to flesh
it out more completely, though.


1 GB = 2^30, 1 TB = 2^40, 1 PB = 2^50, 1 EB = 2^60
NYR = Not Yet Released

OS/hardware        64-bit C    max    max    NFS    info
            datatype    par-    file    V3    updated
                    tition    size    sup
                    size    (bytes)
UniCOS (Cray vector)    int, long    ?    8 EB?    ?    8/94
ConvexOS        long long    1 TB    1 TB    N    9/94
Alpha AXP OSF/1 V3.0
    AFS        long        128 GB    16 TB    8/94    9/94
Paragon OSF/1        ?        8 EB*    8 EB    N    2/95
UTS (Amdahl)        ?        ?    8 EB?    ?    8/94
HP/UX 9 (HP 9xxx)    ?        4 GB    ?    ?    8/94
Silicon Graphics
    IRIX 5.2 EFS    long long    8 GB    2 GB    N    9/94
    IRIX 6.0 EFS    long        8 GB    2 GB    N    9/94 (NYR)
    IRIX 5.3 XFS    ?long long?    ?    ?TB?    Y    9/94 (NYR)
AIX (IBM RS/6000)
    4.1 JFS        long long    64 GB    2 GB    N    8/94
Solaris 2.x (Sun Sparc)    long long    1 TB    2 GB    (soon?)    9/94
BSD 4.4            long long    ?    8 EB?    ?    8/94
Linux            long long    1 TB    2 GB    ?    9/94
DG/UX 5.4        long long    2 TB    2 GB    ?    9/94
Alliant Concentrix    long long    ?>2 GB    ?>2 GB    N    9/94 (dead)

* The Paragon PFS (Parallel File System), as I understand it, parallelizes
access to the files; each partition striped across is limited to 2GB, so
really the max partition size is 2GB * # of disks that can be attached.

A slightly more detailed
description of certain implementations
is available with the WWW version.

In addition, the HPSS (see above) supports large files, as does
Unitree (though the Unitree interface to them is limited).

------------------------------
Subject: [7.9] Non-Unix Large File Systems
From: File Systems

(info about non-Unix large FSes also welcome; SHMO)

OpenVMS (any version) supports 2TB files (32-bit unsigned block
number, 9-bit offset) through its RMS interface (still limited to 2GB
through the C run-time library), but file systems are limited to ~7GB
(as of Open AXP 1.5 and OpenVMS VAX 6.0 the max volume size has been
bumped to 1 TB). (from a friend, rdv, 94/8/26, and Rod Widdowson,
Filesystems group, OpenVMS engineering, Scotland).

------------------------------
Subject: [8] (Device) Interfaces
From: (Device) Interfaces

There is a new web site with lots of info at
http://www.cit.ac.nz/smac/cbt/hwsys/bussys/default.htm (rdv, 96/2/21)
Looks like it's class notes, so no idea how long it will stay up.
Don't forget to see http://www.cmpcmm.com/cc/.

------------------------------
Subject: [8.1] SCSI {Full}
From: (Device) Interfaces

SCSI is the Small Computer System Interface. It is standardized by
ANSI X3T9.2. It is mostly aimed at storage devices, with command sets
defined for disks, tapes, and autochangers, but also includes
communications devices, printers, and scanners.

It's daisy-chained, with a maximum of eight devices (including the
host computer) on a single narrow bus (there are non-standard schemes
for 16 devices on a wide bus). Any device can be an initiator, so it's
possible to use the bus for sharing devices between hosts, provided
your software can manage it.

See also the newsgroup comp.periphs.scsi, especially for "How do I
hook up a Brand X diskdrive to my Atavachron 9000 PDA?" type
questions.

There is also an FTP site for some working documents for the SCSI-3
committees and other X3T10 documents. See ftp://ftp.symbios.com or
ftp.hmpd.com.

You'll find good info at http://www.symbios.com/x3t10/ and at
http://www.scsita.com.

------------------------------
Subject: [8.1.1] Single ended vs differential
From: (Device) Interfaces

This distinction is at the eletrical signalling level. However,
single-ended is limited to total bus lengths of 6.0 meters, while
differential can go up to 25 meters (SCSI-II). Differential is
generally more robust to noise and cross-talk, but the bus drivers are
more expensive. In theory no difference in transfer speed or
capabilities, but in practice the added noise margin could mean higher
_reliable_ transfer rates on your system, especially if your bus is
long.

Most disk drives and most low-end products are available only with a
single-ended interface. A few devices are available with either as a
purchase option, and a few are switchable by the user.

The cables and connectors are the same for both, though the pinouts
are (naturally) somewhat different.

Plugging a single-ended device into a running differential bus or
vice-versa may result in damage to one or more devices. Most newer
devices have fuses or protection circuits utilitizing the DIFFSENSE
signal to prevent device damage.

There are now recommended icons used to distinguish between the two:


single-ended                    differential
  /\                             //\
 /  \                           //  \
<   --                         <<   --
 \  /                           \\  /
  \/                             \\/

Converters do exist that will allow you to hook up single-ended
devices to a differential bus and vice-versa. People who have used
them say they work great, but in theory they shouldn't work :-).  As I
understand it, changing the signalling introduces delays in some of
the control signals that means that some devices could miss certain
signal transitions. The best advice is to borrow one and try it, and
see if it works in your system. One company's name is Paralan,
[phone].

------------------------------
Subject: [8.1.2] Asynchronous vs Synchronous Transfers
From: (Device) Interfaces

Asynchronous transfers mean that every single byte must be
acknowledged before the next can be transfered. Synchronous means that
the device sending data can drop a series of transfers onto the bus,
toggling REQ or ACK (as appropriate), and then sit back and wait for
the corresponding pulses to return from the other device.

Async transfers, involving much more waiting, are correspondingly
slower. 2-4 MB/sec are good values for async transfers.

Sync transfer speeds are established during a negotiation between the
initiator and target, but devices are not required to use the full
speed they negotiate for. This speed represents the maximum burst rate
your device will use. Common values are 5 and 10 MB/sec.

In practice, virtually every modern device supports synchronous
transfers, but some implementations are better than others.

------------------------------
Subject: [8.1.3] SCSI-I vs SCSI-II vs SCSI-III
From: (Device) Interfaces

SCSI (now commonly known as SCSI-I) was the original 1986 standard,
X3.131-1986.  It specified the electrical level and some of the
mid-layer issues involving messages and packet structure, but (I
believe, my memory's bad) didn't formalize the Common Command Set
(CCS), that was done independently. It supported a maximum burst rate
of 5 MB/sec. on an 8-bit bus.

ADDITIONAL INFORMATION

Consult the SCSI standards documents, and the manuals for the device you
are working with for more information.  The "SCSI 1" specification
document is called SCSI Specification, ANSI X3T9.2/86-109. Also of
interest is the Common Command Set specification document SCSI CCS
Specification, ANSI X3T9.2/85-3

SCSI-II received final approval in early 1994, but has been a de facto
standard for several years. The CCS was standardized for a variety of
different types of peripherals. The max allowable transfer rate was
raised to 10 MT/s (see below). A 16-bit bus (Wide SCSI) and 32-bit bus
(double-wide SCSI) are specified (see below).

SCSI-III is the latest effort, and involves more cleanly separating
the functionality into layers; the command layer is defined
independently from the physical layer. In addition to the traditional
parallel cable, there are efforts going on to define physical layers
for Fibre Channel and a more generic Serial SCSI. Thus, there will be
no SCSI-IV; only the individual pieces will be updated as necessary.

------------------------------
Subject: [8.1.4] Fast-Wide SCSI
From: (Device) Interfaces

The max allowable transfer rate was raised to 10 MT/s (mega-transfers
per second) in SCSI-2, referred to as Fast SCSI. Note that this is NOT
required, devices running at ANY speed below that may claim to be
SCSI-II compliant! Fast implies SCSI-II, not the other way around!
Fast Narrow is thus 10 MB/sec. Both the initiator (computer) and
target (peripheral) must support fast transfer for it to be of any
use, but intermixing fast and slow devices on a bus presents no
operational problems (only performance ones).

A 16-bit bus (Wide SCSI) and 32-bit bus (double-wide SCSI) are
specified in SCSI-2. The wide busses require the use of a second cable
in SCSI-2. The first cable is 50 pins, known as the A cable; the 2nd
is 68 pins, known as the B cable. I know of no one actually using
32-bit SCSI, but it would also run on an A/B cable pair. Slow (or
Normal) Wide is thus 5 MT/s * 2 Bytes/T, 10 MB/sec. Fast Wide is 20
MB/sec. Fast Double Wide would be 40 MB/sec.

In the SCSI-3 physical layer spec (SCSI-PH), a single 68-pin cable,
known as the P cable, is allowable for 8 or 16-bit busses. This is the
option most people who have implemented Wide SCSI have chosen for the
cabling, even though their upper layer is generally SCSI-2.

There is a small movement (heard here on the net occassionally) to
promote an Ultra-SCSI high-speed bus, with a burst rate of something
like 20 MT/sec on very short cables. At present it is unclear what
will happen to this effort. There is also talk, in conjunction with a
change to low-voltage differential signalling, to go to 40MT/sec.

------------------------------
Subject: [8.1.5] Shared Busses / Performance {Brief}
From: (Device) Interfaces

Also known as, "It's only a 500KB/sec. tape drive, why do I care if
the burst rate is only 2 MB/sec.?" or gets good marks for "plays well
with others".

Most of this is relevant to all shared busses, not just SCSI.
burst v. sustained performance, disconnect, command overhead, etc.

------------------------------
Subject: [8.1.6] Cabling/Hot Plugging {Brief}
From: (Device) Interfaces

Nominally not supported.

------------------------------
Subject: [8.1.7] Third Party Transfers/Separation of Control & Data Paths {Brief}
From: (Device) Interfaces

SCSI-2 has commands that support third-party copying of data; one
initiator tells device A to copy to device B. I don't know of any
devices actually using this.

Separation of control & data paths is a popular topic these days; can
somebody comment on whether or not SCSI-3 supports this? I don't think
so. (SHMO)

------------------------------
Subject: [8.2] IDE {Brief}
From: (Device) Interfaces
        PC use
        Does not support overlapped I/O.

------------------------------
Subject: [8.3] IPI {None}
From: (Device) Interfaces

------------------------------
Subject: [8.4] HIPPI {Brief}
From: (Device) Interfaces

32-bit transfers at 25 MT/sec., 100 MB/sec. High Performance Parallel
Interface is a unidirectional channel, i.e. you have to have an OUT
cable and and IN cable for bidirectional transfers (you could have
just one, if it's a read-only device like a scanner or write-only like
a frame buffer). HiPPI is not a shared bus, but its frames can be
switched through a crossbar switch (Network Systems is the premiere
vendor).

HiPPI is used for supercomputer-to-supercomputer networking (TCP/IP,
no less), for RAID arrays (from Maximum Strategy, IBM and others),
tape drives (Sony ID-1 drive), frame buffers and increasingly
workstations (SGI and IBM support HiPPI, and 3rd-party Sbus cards
exist for Sun).

Due partly to the high overhead of HiPPI connections, many devices
have elected to separate the control path from the data path. A common
control path in that case is ethernet.

Good resources from the HiPPI Networking Forum on the web at
http://www.esscom.com/hnf/.

------------------------------
Subject: [8.4.1] HIPPI-6400 {Brief}
From: (Device) Interfaces

An effort aimed at reaching 6400 Mbps (800 Mbytes/sec.) around the end
of 1996.

From rev 0.15 of the HIPPI-6400-PH specification, dated March 4, 1996,
ftp'ed from ftp.network.com:X3T11/hippi/hippi-6400-ph_0.15.ps.

Looks like the copper interface will be a cable with 44 micro-coax
conductors, 22 in each direction. That's 16 data, 4 control, clock,
and frame. A micro-packet is 32 data bytes and 64 bits of control
information. I guess this means they're planning on 400 Mbps on each
data line. The fiber variant uses 12 multimode fibers (in each
direction, I presume, though it doesn't seem to say that): 8 data + 2
control + frame + clock, so presumably 800 Mbps on each fiber. Cable
lengths in both cases TBD.

------------------------------
Subject: [8.5] Ultranet {Brief}
From: (Device) Interfaces

Fiber to the host, a hub with a backplane running at a total rate of
~1Gbps.

------------------------------
Subject: [8.6] Ethernet {Brief}
From: (Device) Interfaces

Generally related to normal inter-host networking, but also used as
a control path for some HiPPI devices. Ampex also uses NetSCSI over
ethernet to control their autochangers. Also, obviously, used for
connecting many servers to their clients. Standard today is 10 Mbps,
100 Mbps (fast ethernet) is becoming more common.

------------------------------
Subject: [8.7] FDDI {None}
From: (Device) Interfaces

------------------------------
Subject: [8.8] Fibre Channel Standard (FCS)
From: (Device) Interfaces

Rich Taborek of Amdahl has created an excellent web page on Fibre
Channel at http://www.amdahl.com/ext/CARP/FCA/FCA.html.

    ftp.network.com     [Has draft Fibre Channel documents]
    playground.1.com  [Has FCSI Fibre Channel Profiles]
(rdv, 95/5/18 from Louis Grantham <[e-mail]>)

Fibre Channel runs over coax or optical fibre (single or multimode),
and even twisted pair.  Fibre Channel comes in two basic forms --
Aribtrated Loop and switched fabric, which aren't (yet)
interoperable.  The host interfaces are rapidly becoming cheaper, but
the switches are still expensive.

Fibre Channel standards define several functional levels, from the
physical interface up to the mapping to upper level functionality,
e.g. how to do SCSI commands over FC.  FC provides several "classes"
of service, including dedicated circuit and acknowledged and
unacknowledged datagrams.  Can also be used for IP.  (rdv, 96/10/28)

------------------------------
Subject: [8.9] ESCONN/SBCON {Brief}
From: (Device) Interfaces

Enterprise Systems CONNect. IBM's new mainframe attach -- fiber, I
believe. The standardized version of this is known as SBCON, and Rich
Taborek has once again created an excellent web page at
http://www.amdahl.com/ext/CARP/SBCON/SBCON.html.

------------------------------
Subject: [8.10] IEEE P1394 (Serial Bus)
From: (Device) Interfaces

Apple's new standard for connecting devices via a high-speed serial
bus. Good info at http://www.skipstone.com. Also some info FTPable at
ftp.apple.com (I think that's where I got those papers.) (rdv,
95/5/15)

After having been somewhat dormant for a while, standards activity on
new versions of 1394 is heating up again.  Faster versions are in the
works, as is a protocol for doing disks across it. (rdv, 96/10/28)

------------------------------
Subject: [8.11] Serial Storage Architecture (SSA)
From: (Device) Interfaces

IBM's new offering in the serial device interface sweepstakes.
Some docs and tentative working standards available on the SCSI ftp
site: ftp.symbios.com:pub/standards/io/ssa The SSA Industry
Association has a web server at http://www.ssaia.org. Disk drives from
Conner and Micropolis, and Pathlight and Adaptec and expected to do
host adapters.

------------------------------
Subject: [8.12] S2I: IEEE P1285 Scalable Storage Interface
From: (Device) Interfaces

Chaired by Martin Freeman, Philips Research, this is an effort to
standardize attaching disk drives directly to a system bus, making the
disk's buffers readable as regular memory to the CPU. Sort of the
opposite of network-attached storage, this couples the storage device
design more closely to the hardware and OS of the host system. See
http://sunrise.scu.edu/P1285Home.html for more info. (rdv, 1995/12/22)

------------------------------
Subject: [8.13] Multibus, Unibus, Mainframe Channels, and other history {None}
From: (Device) Interfaces

------------------------------
Subject: [9] Other
From: Other
------------------------------
Subject: [9.1] Video vs Datagrade tapes {brief, 5/94}
From: Other
        cost vs reliability
        Are datagrade really more reliable?
        Warrantee of drive
        Cleaning cycle of drive
        Headlife of drive


------------------------------
Subject: [9.2] Compression
From: Other

See the comp.compression FAQ, and don't believe everything a vendor
tells you. 2x compression is the standard going rate for lossless
compression of arbitrary data, though some vendors claim 2.5 or 3x.
Your mileage will vary with your data type.

Compressing tape drives are common, but for disks and other block
devices I don't know of anything being done. The unpredictability of
the compression ratio generally makes it inappropriate for devices
that need fixed capacities and addresses.

Online compression of files can be accomplished by hand using
utilities such as gzip and Unix compress. Some systems support
software compression of files in the file system software, and will
transparently compress and decompress files as needed. Stacker for PCs
is one example; for Unix-like systems this seems to be common research
for object-oriented file systems (including the GNU Hurd), but I don't
know of any production versions offhand (SHMO).

Compression may make your data more vulnerable to errors. A single
error early in a compressed stream of data can render the entire data
stream unreadable.

------------------------------
Subject: [10] Benchmarking
From: Benchmarking

See the comp.benchmarks FAQ, and don't believe everything a vendor
tells you.

There's a good paper on a new I/O benchmarking technique that also
covers the pitfalls of I/O benchmarking in the Nov. '94 ACM
Transactions on Computer Systems -- "A New Approach to I/O Performance
Evaluation -- Self-Scaling I/O Benchmarks, Predicted I/O Performance",
Peter Chen and David Patterson.

Bonnie, IOZONE, IOBENCH, nhfsstone, one of the SPECs (SFS), are all
useful for measuring I/O performance. There is also a program called
BENCHMARK available from [e-mail] -- apparently a
standardized set of scripts to test remote access to mass storage
systems.

In particular, note that based on a discussion here recently (8/96),
it appears that some magazines (who ought to know better) are using
HDT BenchTest as a disk drive performance measure, with the I/O sizes
set so small that the disk drive cache is covering them all, resulting
in anomalously high data rates (especially write rates).

http://home.hkstar.com/~tamws/comp/bench/hdbench.htm is the start of a
reasonable-looking benchmark for PC hard drives (posted by
[e-mail], 9/96)

 ==== SPEC SFS ====

SPEC's System-level File Server (SFS) workload measures NFS server
performance. It uses one server and two or more "load generator"
clients.

SPEC-SFS is not free; it costs US$1,200 from the SPEC corporation.
There's a FAQ about SPEC posted sometimes in comp.benchmarks.

------------------------------
Subject: [11] Mass Storage Conferences
From: Mass Storage Conferences

There are two main academic conferences devoted specifically to mass
storage (in addition to, of course, the supercomputer and OS
conferences, and interesting stuff in databases, optical
conferences, Usenix, SOSP...).

NASA Goddard Space Flight Center and the IEEE run two conferences, in
an 18-month or so alternating pattern.

You'll find my notes on the letest Goddard conference at
http://www.isi.edu/~rdv/conferences/goddard96.html.

The contact for the NASA Mass Storage Conference (Sept. 17-19, 1996):


Jorge Scientific Corporation
7500 Greenway Center Drive
Suite 1130
Greenbelt, MD USA 20770
tel[phone]
fax[phone]


or if that fails email [e-mail] or
[e-mail]. There is some info available on the web at
http://esdis.gsfc.nasa.gov/msst/msst.html.

Also, the latest IEEE was in September '95:

* The 14th IEEE Mass Storage Symposium was September 11-14, 1995 at
  Monterey, CA.  More info from Bernie O'Lear ([e-mail]) or
  Sam Coleman ([e-mail]).

Also of interest, there are the conferences on Very Large Database
Systems. I have a reference somewhere...

Interesting material shows up in the SPIE conferences.

------------------------------
Subject: [11.0.1] THIC Tape Head Interface Committee {Brief, New}
From: Mass Storage Conferences

I would like to bring to your attention the THIC Home Page at the URL
http://www.thic.org/thic/ and its anonymous ftp archives at the URL
ftp://ftp.uu.net/vendor/THIC/

THIC started out in the early 70's as the Tape Head Interface Committee
under the auspices of the DoD, but has since grown and expanded to embrace
most data recording technologies.  THIC has been meeting four times a year,
alternating between the east and west coasts.  The last meeting was in
Seattle WA on Jan 21 and 22, 1997, and the next will be on April 22 and 23
at the DoubleTree in Tysons Corner VA.  The papers range from marketing,
new product announcement and discussion, to the problems of the various
recording technologies.  Since October 1995, I have been trying to collect
as many of the papers as I could from each of the meetings and have been
placing them in Adobe PDF on the THIC archives at ftp.uu.net.  I also
maintain a no-frills home page where the agenda is displayed, with links to
papers which are available in the archives. (P.C. Hariharan, 97/2)

------------------------------
Subject: [12] MTBF (Mean Time Between Flareups, er, Failures)
From: MTBF (Mean Time Between Flareups, er, Failures)

There is a short FAQ-like document available from IBM at
http://www.storage.ibm.com/storage/oem/tech/mtbf.htm. No math for the
statistically inclined, but explains in clear prose what IBM at least
means when they say MTBF.

I will also note that, for a complex but reparable system such as an
autochanger, each subsystem may have a separate MTBF and a different
lifetime, which may be combined to give one figure for the unit as a
whole.

Here is a reasonably understandable, but somewhat long, description of
MTBF. Thanks to Kevin Daly (president of Odetics, [e-mail])
wrote in 10/95 for this FAQ. After some waffling, I've included the
whole thing, despite its length.

===============================================================


M T B F

In order to understand MTBF (Mean Time Between Failures) it is best to
start with something else -- something for which it is easier to
develop an intuitive feel.  This other concept is failure rate which
is, not surprisingly, the average (mean) rate at which things fail.  A
"thing" could be a component, an assembly, or a whole system.  Some
things -- rocks, for example -- are accepted to have very low failure
rates while others -- British sports cars, for example -- are (or
should be) expected to have relatively high failure rates.

It is generally accepted among reliability specialists (and you,
therefore, must not question it) that a thing's failure rate isn't
constant, but generally goes through three phases over a thing's
lifetime.  In the first phase the failure rate is relatively high, but
decreases over time -- this is called the "infant mortality" phase
(sensitive guys these reliability specialists).  In the second phase
the failure rate is low and essentially constant -- this is
(imaginatively) called the "constant failure rate" phase.  In the
third phase the failure rate begins increasing again, often quite
rapidly, -- this is called the "wearout" phase.  The reliability
specialists noticed that when plotted as a function of time the
failure rate resembled a familiar bathroom appliance -- but they
called it a "bathtub" curve anyway.  The units of failure rate are
failures per unit of "thing-time"; e.g. failures per machine-hour or
failures per system-year.

What, you may ask, does all this have to do with MTBF?  MTBF is the
inverse of the failure rate in the constant failure rate phase.
Nothing more and nothing less.  The units of MTBF are (or, should be)
units of "thing-time" pre failure; e.g. machine-hours per failure or
system-years per failure but the "thing" part and the "per failure"
part are almost always omitted to enhance the mystique and confusion
and to make MTBF appear to have the units of "time" which it doesn't.
We will bow to the convention of speaking of MTBF in hours or years --
but we all know what we really mean.

What does MTBF have to do with lifetime?  Nothing at all!  It is not
at all unusual for things to have MTBF's which significantly exceed
their lifetime as defined by wearout -- in fact, you know many such
things.  A "thirty-something" American (well within his constant
failure rate phase) has a failure (death) rate of about 1.1 deaths per
1000 person-years and, therefore, has an MTBF of 900 years (of course
its really 900 person-years per death).  Even the best ones, however,
wear out long before that.

This example points out one other important characteristic of MTBF --
it is an ensemble characteristic which applies to populations (i.e.
"lots") of things; not a sample characteristic which applies to one
specific thing.  In the good old days when failure rates were
relatively high (and, therefore, MTBF relatively low) this
characteristic of MTBF was a curiosity which created lively (?) debate
at conventions of reliability specialists (them) but otherwise didn't
unduly bother right-thinking people (us).  Things, however, have
changed.  For many systems of interest today the required failure
rates are so low that the MTBF substantially exceeds the lifetime
(obviously nature had this right a long time ago).  In these cases
MTBF's are not only "not necessarily" sample characteristics, but are
"necessarily not" sample characteristics.  In the terms of the
reliability cognoscenti, failure processes are not ergodic (i.e. you
can't blithely trade population statistics for time statistics).  The
key implication of this essential characteristic of MTBF is that it
can only be determined from populations and it should only be applied
to populations.

MTBF is, therefore an excellent characteristic for determining how
many spare hard drives are needed to support 1000 PC's, but a poor
characteristic for guiding you on when you should change your hard
drive to avoid a crash.

MTBF's are best determined from large populations.  How large?  From
every point of view (theoretical, practical, statistical) but cost,
the answer is "the larger, the better".  There are, however, well
established techniques for planning and conducting test programs to
develop specified levels of confidence in a thing's MTBF.
Establishing an MTBF at the 80% confidence level, for example, is
clearly better, but much more difficult and expensive, than doing it
at a 60% confidence level.  As an example, a test designed to
demonstrate a thing's MTBF at the 80% confidence level, requires a
total thing-time of 160% of the MTBF if it can be conducted with no
failures.  You don't want to know how much thing-time is required to
achieve reasonable confidence levels if any failures occur during the
test.

What, by the way, is "thing-time"?  An important subtlety is that
"thing-time" isn't "clock time" (unless, of course, your thing is a
clock).  The question of how to compute "thing-time" is a critical one
in reliability engineering.  For some things (e.g.  living thing) time
always counts but for others the passage of "thing-time" may be highly
dependent upon the state of the thing.  Various ad hoc time
corrections (such as "power on hours" (POH)) have been used, primarily
in the electronics area.  There is significant evidence that, in the
mechanical area "thing-time" is much more related to activity rate
than it is to clock time.  Measures such as "Mean Cycles Between
Failures (MCBF)" are becoming accepted as more accurate ways to assess
the "duty cycle effect".  Well-founded, if heuristic, techniques have
been developed for combining MCBF and MTBF effects for systems in
which the average activity rate is known.

MTBF need not, then be "Mysterious Time Between Failures" or
"Misleading Time Between Failures", but an important system
characteristic which can help to quantify the suitability of a system
for a potential application.  While rising demands on system integrity
may make this characteristic seem "unnatural", remember you live in a
country of 250 million 9- million-hour MTBF people!

===================================================================
Kevin C. Daly
President
ATL Products
[e-mail]
[phone]


------------------------------
Subject: [13] Mass Storage Reports
From: Mass Storage Reports

There are a number of consultants who also write regularly updated
in-depth reports (and sometimes post here) about various aspects of
the mass storage market; if you're going to get into this business or
are planning on spending many thousands or millions of dollars on
equipment, talking to one of them might be a good idea.

Sanjay Ranade ([e-mail]) is one of the ones who both writes
and posts here (he also has a couple of reasonably-priced books about
mass storage). Infotech's reports include HSM, network backup,
magtape and libraries.

Others include Disk/Trend (Mountain View, CA, 405-961-6209)
http://www.disktrend.com (good info there) and Freeman Reports
(805-963-3853).

Strategic Research Corporation has numerous white papers and good
links available at http://www.sresearch.com, including networked
storage. Some of them seem biased in particular directions, so caveat
emptor.

------------------------------
Subject: [14] Network-Attached Peripherals {Brief}
From: Network-Attached Peripherals {Brief}

Coming soon. My own research is in this area; if you're lucky you
might find some pointers by going through my home page
http://http://alumni.caltech.edu/~rdv/. Contributions welcome.

Look for "A Brief Survey of Current Work on Network-Attached
Peripherals" in the January '96 ACM Operating Systems Review, by yours
truly. An expanded, updated version is available on the web at
http://www.isi.edu/~rdv/netstation/nap-research/. (rdv, 96/1/22)

http://www.cs.cmu.edu/Web/Groups/PDL/ is Garth Gibson's Parallel Data
Lab, where they're doing excellent work on network-attached storage
devices.

At Lawrence Livermore, they're doing a network-attached RAID array to
integrate into HPSS; see
http://www.llnl.gov/liv_comp/siof/siof-nap.html.

The ViewStation work at MIT,
http://tns-www.lcs.mit.edu/tns-www-home.html is concentrating on
ATM-attached peripherals, using ATM as a system-area network.

The Netstation project http://www.isi.edu/netstation/ (which I work on
at ISI) is focusing on IP-connectible peripherals, using a gigabit
network as the system backplane.

------------------------------
Subject: [15] Other References
From: Other References

------------------------------
Subject: [15.1] Print
From: Other References


        Computer Technology Review magazine, 310/208-1335, free to some.
        Electronic News, weekly, 800/722-2346.
        MacWeek, June 7, 1993, Page 36+
    IEEE Computer had a full issue in March 94 on I/O subsystems

There are also two books by Sanjay Ranade ([e-mail]), who
posts here occassionally. One is _Mass Storage Technologies_
(1991ish?), the other, newer one is _Mass Storage Systems_. I've read
the first one, it's a little short on detail but a good overview.

------------------------------
Subject: [15.2] Web
From: Other References

http://www.cmpcmm.com/cc/ standards and tons of info.

http://www.nml.org performance reports, media surveys, etc. Goes into a
lot of detail on topics such as archival stability.

http://www.yahoo.com/Business/Corporations/Computers/Peripherals/Storage/
lists some resellers and manufacturers of storage.

http://theref.c3d.rl.af.mil has good information about PC hardware,
including old interfaces, floppies, controllers, etc. It has a LONG
list of specs for hard drives.

http://www.cs.yorku.ca/People/frank/Welcome.html also has good info on
hard drives and CD-ROM drives.

http://www.sresearch.com/search/105008.htm lists storage products and
market projections.

------------------------------
Subject: [15.3] Newsgroups
From: Other References

You're in the primary one (comp.arch.storage).  You'll also find info
in the groups on SCSI, PC hardware, and specific operating systems.
I'll try to add pointers to their FAQs soon.

The FAQ for comp.sys.ibm.pc.hardware.storage can be found at
http://thef-nym.sci.kun.nl/~pieterh/storage.html.

------------------------------
Subject: [15.4] Research Papers
From: Other References

I'm collecting reviews and a list of papers now, I expect to add it in
a few weeks. Contributions/suggestions welcome.

------------------------------
Subject: [16] ORIGINAL CALL FOR VOTES
From: ORIGINAL CALL FOR VOTES

NAME:
          comp.arch.storage

STATUS:
          unmoderated


DESCRIPTION:

        storage system issues, both software and hardware


CHARTER:

To facilitate and encourage communication among people interested in computer
storage systems.  The scope of the discussions would include issues relevant
to all types of computer storage systems, both hardware and software.  The
general emphasis here is on open storage systems as opposed to platform
specific products or proprietary hardware from a particular vendor.  Such
vendor specific discussions might belong in comp.sys.xxx or comp.periphs.
Many of these questions are at the research, architectural, and design levels
today, but as more general storage system products enter the market,
discussions may expand into "how to use" type questions.


RATIONALE:

As processors become faster and faster, a major bottleneck in computing
becomes access to storage services:  the hardware - disk, tape, optical,
solid-state disk, robots, etc., and the software - uniform and convenient
access to storage hardware.  A far too true comment is that "A supercomputer
is a machine that converts a compute-bound problem into an I/O-bound
problem."  As supercomputer performance reaches desktops, we all experience
the problems of:

o       hot processor chips strapped onto anemic I/O
                architectures
o       incompatable storage systems that require expensive
                systems integration gurus to integrate and
                maintain
o       databases that are intimately bound into the quirks of an
                operating system for performance
o       applications that are unable to obtain guarantees on when
                their data and/or metadata is on stable storage
o       cheap tape libraries and robots that are under-utilized
                because software for migration and caching to
                disk is not readily available
o       nightmares in writing portable applications that attempt
                to access tape volumes

This group will be a forum for discussions on storage topics including the
following:

>1.     commercial products - OSF Distributed File System (DFS)
        based on Andrew, Epoch Infinite Storage Manager and
        Renaissance, Auspex NS5000 NFS server, Legato
        PrestoServer, AT&T Veritas, OSF Logical Volume Manager,
        DISCOS UniTree, etc.
>2.     storage strategies from major vendors - IBM System
        Managed Storage, HP Distributed Information Storage
        Architecture and StoragePlus, DEC Digital Storage
        Architecture (DSA), Distributed Heterogeneous Storage
        Management (DHSM), Hierarchical Storage Controllers, and
        Mass Storage Control Protocol (MSCP)
>3.     IEEE 1244 Storage Systems Standards Working Group
>4.     ANSI X3B11.1 and Rock Ridge WORM file system standards
        groups
>5.     emerging standard high-speed (100 MB/sec and up)
        interconnects to storage systems: HIPPI, Fiber Channel
        Standard, etc.
>6.     POSIX supercomputing and batch committees' work on
        storage volumes and tape mounts
>7.     magnetic tape semantics ("Unix tape support is an
        oxymoron.")
>8.     physical volume management - volume naming, mount
        semantics, enterprise-wide tracking of cartridges, etc.
>9.     models for tape robots and optical jukeboxes - SCSI-2,
        etc.
>10.    designs for direct network-attached storage (storage as
        black box)
>11.    backup and archiving strategies
>12.    raw storage services (i.e., raw byte strings) vs.
        management of
        structured data types (e.g. directories, database
        records,...)
>13.    storage services for efficient database support
>14.    storage server interfaces, e.g., OSF/1 Logical Volume
        Manager
>15.    object server and browser technology, e.g. Berkeley's
        Sequoia 2000
>16.    separation of control and data paths for high performance
        by removing the control processor from the data path;
        this eliminates the requirements for expensive I/O
        capable (i.e., mainframe) control processors
>17.    operating system-independent file system design
>18.    SCSI-3 proposal for a flat file system built into the
        disk drive
>19.    client applications which bypass/ignore file systems:
        virtual memory, databases, mail, hypertext, etc.
>20.    layered access to storage services - How low level do we
        want device control?  How to support sophisticated, high
        performance applications that need to bypass the file
        abstraction?
>21.    migration and caching of storage objects in a distributed
        hierarchy of media types
>22.    management of replicated storage objects
        (differences/similarities to migration?)
>23.    optimization of placement of storage objects vs. location
        transparency and independence
>24.    granularity of replication - file system, file, segment,
        record, etc.,
>25.    storage systems management - What information does an
        administrator need to manage a large, distributed storage
        system?
>26.    security issues - Who do you trust when your storage is
        directly networked?
>27.    RAID array architectures, including RADD (Redundant
        Arrays of Distributed Disks) and Berkeley RAID-II HIPPI
        systems
>28.    architectures and problems for tape arrays - striped tape
        systems
>29.    stable storage algorithm of Lampson and Sturgis for
        critical metadata
>30.    How can cheap MIPS and RAM help storage? -  HP DataMesh,
        write-only disk caches, non-volatile caches, etc.
>31.    support for multi-media or integrated digital continuous
        media (audio, video, other realtime data streams)

This group will serve as a forum for the discussion of issues which do not
easily fit into the more tightly focused discussions in various existing
newsgroups.  The issues are much broader than Unix (comp.1.*, comp.os.*),
as they transcend operating systems in general.  Distributed computer systems
of the future will offer standard network storage services; what operating
system(s) they use (if any) will be irrelevant to their clients.  The
peripheral groups (comp.periphs, comp.periphs.scsi) are too hardware oriented
for these topics.  Several of these topics involve active standards groups
but several storage system issues are research topics in distributed systems.
In general, the standards newsgroups (comp.std.xxx) are too narrowly focused
for these discussions.


--------------------------------------------------------
------------------------------
Subject: [17] Original Author's Disclaimer and Affiliation:
From: Original Author's Disclaimer and Affiliation:

This information is believed to be reasonably accurate although I do
not verify every submission.  Neither the United Stages Government nor any
agency thereof, nor any of their employees, makes any warranty, express or
implied, or assumes any legal liability or responsibility for the accuracy,
completeness, or usefullness of any informatin, apparatus, product, or
process disclosed, or represents that its use would not infringe privately
owned rights.  Reference herein to any specific commercial product, process,
or service by trade name, trademark, manufacturer, or otherwise, does not
necessarily constitute or imply its endorsement, recommendatin, or favoring
by the United States Government or any agency thereof.  The views and
opinions of authors expressed herein do not necessarily state or reflect
those of the United States Government or any agency thereof.


---
Joseph Stith, [e-mail], 708/840-3846
Assistant to the Computing Division Head -- IRM Planning
Computing Division, Fermilab, PO Box 500, MS 120, Batavia, IL 60510


------------------------------
Subject: [18] Copyright Notice
From: Copyright Notice

This compilation of material is copyright Rod Van Meter,
[e-mail]. Permission is granted to copy this material,
provided this copyright notice is retained. The contents are not to be
significantly modified without the express written consent of the
author.

This is just to keep the various authors of this material from being
substantially misquoted or abused, not to restrict use of the
information.

Permission to include this FAQ in published compilations (CD-ROM or
book) will be granted upon direct request.

------------------------------
Subject: [19] Additional Topics to be added
From: Additional Topics to be added

File Systems: Unix, IBM, VMS, Tops-20, Extent-based, Amiga, Mac
(resource & data forks)
FTP Sites
Volume Sets & Partitions
Important People/Mass Storage History
Books & Other Publications
Principles for Evaluating New Technologies
Performance Evaluation
  cacheing
  seek time measurement
  concurrent operations
  queueing theory
Head Lifetime
Versioning in File Systems
Managing Risk
Media Migration/Managing Change
Physical v. Logical Addressing (seek optimizations, etc.)
Channels v. Busses
Intelligent Storage Subsystems
DEC's HSC-50 and star cluster for VAXen
Mainframe & Supercomputer I/O controllers
Security
The broadcast and home audio/video / mass storage connection
Databases and Mass Storage
File System Research: watchdogs, named pipes, compressing FSes
The naming problem: Prospero
Distributed Locking & Update
Content-Addressable Storage & Other Unusual Ideas
The old film-storage system Sam Coleman talks about
Byte Ordering
Supercomputer Storage
Companies: Adstor, Avastor
I/O Benchmarks
User file systems
System CPU & bus loads for file system work
Memory-Mapped Files
Persistent Object Systems & their files
The VFS layer in Unix
What to look for in a backup product
Offsite Storage v. Network Backup
Test Equipment -- SCSI & HiPPI Analyzers
(reorganize along small user/large user/developer lines?)
(need to date every entry if possible)
terminology