HSM systems transparently migrate files from disk to optical disk and/or magnetic tape, usually robotically accessible. Then when files are accessed by a user, they transparently move them back to disk.
Watch for maximum file size limitations, sometimes limited by the size of the media used, sometimes by the server's OS, and sometimes neither.
Some offer integrated backup. Some will manage multiple copies of files for data reliability.
Some offer integrated migration from other systems (ie, file servers and/or workstations) to the central location disks, then to the central location robotics. This generally requires changes to the on-disk file system format on the migration clients.
An item to watch for is that the file management may be exactly like Unix -- that is, all files appear to be online, and once they're deleted, they're gone forever, even though the data may still be on tape.
All of the subsections here are Unix-compatible (in various flavors) unless indicated otherwise.
Additional Information:
See also _DEC Professional_, February 1993, Page 40 and _Client/Server Today_, Dec. '94, p. 60.
The System-Managed Storage Guide by Howard W. Miller, $225 for first copy, $75 for additional copies for same company available from The Information Technology Institute, 136 Orchard Street, Byfield, Massachusetts, 01922-1605.
(stith@fnal.gov)
Thomas Woodrow did an evaluation of NAStore, FileServ, DMF and Unitree
in 1993. It can be obtained through
www.nas.nasa.gov
or the Proc. 3rd NASA Goddard Conference on Mass Storage Systems and
Technologies, Oct. 1993, pp. 187--216. Somewhat dated now but
excellent methodology for comparing HSMs.
The uncle of UNIX HSMs. Developed primarily at Lawrence Livermore
National Laboratories. Commercialized by a company called DISCOS,
then sold to OpenVision. UniTree was sold to UniTree Software in
December, 1994. See www.unitree.com.
Many versions exist on different hardware platforms, including a National Storage Lab (NSL) UniTree commercialized by IBM - Fed Systems. It's also available on SGI, Convex, and Amdahl hardware, at least.
See also "Epoch vs Unitree" below
For Convex, try
Jim Wilson 214-497-3085 jrwilson@convex.com Business Development Data Management Applications Convex Computer Corporation
For most other platforms, call Open Vision at (800)223-OPEN or (510)426-6400.
New info:
The latest release of UniTree, V1.9.1, has the following changes:
(Note: this evaluation is old, and should be taken with a grain of salt. rdv, 3/96)
(6/93) We just bought both last year. We bought an Epoch I with the 20 GB EO and 327 GB worm. We will be upgrading it to an Epoch II soon. We also bought Unitree from Titan to run on a Silicon Graphics server and hook up to the STK 3480 silo. We hope to add more silos eventually.
Unitree is licensed based on storage capacity while Epoch is not. There may be an exception to this - STK just began reselling Epoch as the front end for their silos and I'm not sure how they handle licensing.
My office mate and I (I handle Epoch, he handles Unitree) have enjoyed comparing the merits/demerits of each over the last year. Comparison in our case is slightly slanted due to the fact that the Epoch has optical disk while the Unitree system has 3480 tape - so some observations have more to do with media advantages/disadvantages.
Unitree
+ Allows large files - can span volumes
+ Allows you to enlarge the cache easily, allows very large
cache
+- Unitree has replaced several UNIX utilities with their own
(FTP, NFS and the file system). This allows certain features to
work but is generally slower and disallows access to the archive when
you are on the server itself - unless you NFS mount!
+ Allows deleted files to be saved for a specified time
+ Allows multiple copies of files to be kept
+ Data is copied to archive soon after creation
+ Unitree runs on several different platforms
In all fairness to Titan, they have been addressing any problems and it has been improving. Epoch too plans to address some of their shortcomings. We are looking forward to growing with both products.
The likelihood that the various flavors of Unitree will standardize depends on what happens with Discos. My guess is that some features/enhancements will be filtered back to the base product released by Discos. Bye...
(bodoh@dgg.cr.usgs.gov, 152.61.192.66, Tom Bodoh, USGS/EROS Data Center, Sioux Falls, SD)
NSL is an industry consortium (American companies only) that has a version of Unitree, and is creating their own new High Performance Storage System.
HPSS, among other features, supports striping of removable media, and full 64-bit files. Some of the work is being done at LLNL, where UniTree was originally developed. The HPSS page is at ORNL.
(rdv,95/1/12)
HIARC HSM runs on Solaris 2.4 and above. Slides in at the vnode
layer. Supports 4mm, 8mm, 3480, DLT, VHS, D-1 and D-2 tape drives,
and appropriate robotics (I don't have a specific list). Removable
media formats are standard (_which_ standard, I don't know). Pricing
from $4k to $25k is reasonable for the functionality. See
www.hiarc.com. (rdv, 97/3/20)
NETstor can be reached at netstor-sales@netstor.com
NETstor, Inc. (formerly Zetaco, Inc.) is a leading provider of hierarchical online mass-storage systems for open systems. Primarily NFS accessable systems with magnetic disks and optical-disk libraries. They have marketing agreements with Digital Equipment Corp, and Hewlett-Packard.
(stith@fnal.gov)
Netstor was bought by Cheyenne, and is now sold by them (lily@access.digex.com, 10/95).
Contact: Steve Wine, Manager, Mass Storage Products, R-Squared, 11211 East Arapahoe Rd, Englewood, CO 80112, 303/799-9292 or FAX 303/799-9297
From Advanced Archival Products. Supports a huge range of devices, autochangers, and operating systems. Block-based movement of data between the hard disk cache and tape or optical tertiary storage. Systems run from a few gigabytes up to at least 12 TB, with prices dependent on capacity. New versions allow multiple cache disks. Slips right in to the VFS layer and looks like a normal Unix file system, with the plusses and minuses that entails. No file versioning or multiple copies yet. File creation is an Achilles' heel on performance. Since it's block based, files can be larger than a piece of media. Separate product DataMgr will migrate files from client machines to the AMASS server automatically (with FS changes, of course).
AMASS is now owned by EMASS, and you can find info at
www.emass.com.
(rdv, 1996/3/27)
Metior (pronounced like meteor) is targetting an incredibly broad market, from laptops with removable media through supercomputers, with prices from $650(!) to $118K. They handle multiple coordinated copies, so off-site backup can be automatic. Can do migration for client machines (with appropriate software licenses and changes to the file system). The hierarchy seems to be extremely flexible, variable on a per-user or per-group basis. Machines without client licenses can mount the Metior FS using NFS. Runs on Suns, SGI, and HP 9000/700. ANT is new, and they've only got a handful of customers so far, but it looks _very_ interesting.
(info from habbott@csn.org, written by rdv, so it's my fault if it's not accurate) (rdv,94/7/7)
More information available on the WWW FAQ version.
Click here to see an email conversation I
had with Hal.
Also see them at anthill.com.
Automated Network Technologies 3333 South Bannock Street, Suite 945 Englewood, CO 80110 USA Phone 303.789.2506 FAX 303.789.2438 Email hal@anthill.com
NAStore is a Unix migrating file system developed by the Numerical Aerodynamic Simulation program at NASA Ames. It is available through NASA's software distribution agency, COSMIC. It currently runs only on Convex with 34x0 cartridges and Storage Tek robots. Looks like a local file system to users of the Convex. Available with source.
Info on NAStore can be found on the web at
chuck.nas.nasa.gov
COSMIC's address is :
University of Georgia
382 East Broad Street
Athens, Georgia, 30602-4272, US
011-706-542-3265
service@cossack.cosmic.uga.edu
For more information on NAStore, contact John Lekashman, lekash@nas.nasa.gov. (info from Bill Ross, bross@nas.nasa.gov, 94/9/15)
Cray Research's Data Migration Facility. The grandaddy of Unix HSM
systems. You can find info on DMF at
www.cray.com, or call +1 612
683 3897 or email crayinfo@cray.com. It's reportedly
running on more than 200 systems, and development is continuing. Large
users are in the hundreds of TB, with millions of files and >1TB/day
through DMF.
Information from: "Storage Management at Cray Research, Inc", Metcalfe, D.J. and Thompson. D. "Data Migration Facility Development Update", Lazatella, T.W. and Bannister, N. Cray User Group, Barcelona, 1996, in press.
(Robert.Bell@dit.csiro.au, 1996/4/2)
From E-Systems. Works with the E-Systems ER-90 (D-2) tape drive and Odetics robots, as well as 3480 with the Storage Tek ACS 4400. Runs on Convexes (only?). Supports multiple copies of files. Retrieves only necessary info from tape to disk before completing request.
Reportedly no longer available on Convex, in beta test on SGI (lily@access.digex.net, 10/95)
Now owned by EMASS, info at
www.emass.com.
They have some agreement with Legent Corporation. OSM runs on Sparc machines, including the Cray Superservers. Price ranges from $500 to $5,000, which is very cheap for HSM. However, it might only be capable of migrating among disks -- I don't see any mention of autochangers. (rdv, 94/12/9)
Supports multiple types of tertiary media (optical, tape) though it
seems to come originally from their work for their own MO jukeboxes.
Supports multiple types of clients. (info from Herbert Volk
More information available on the WWW FAQ version. Click here.
Used to be ASC (Advanced Systems Concept) before being bought by
Platinum. Runs on SunOS, HP, and Domain/OS. Supports numerous optical
jukeboxes. See www.platinum.com. (rdv, 96/4)
PLATINUM technology, inc. 1815 South Meyers Road Oakbrook Terrace, IL 60181 1-800-442-6861 -or- 708/620-5000 e-mail: info@platinum.com
www.lsci.com describes their Solaris-based HSM product. Only one
computing platform, but a reasonably broad range of mid- to high-end
peripherals and robotics supported, from little Exabyte autochangers
to the IBM 3494 and STK silos. (rdv, 96/7/23)
This list is adapted from _Client/Server Today_, Dec. '94, with some of my own additions. All the phone numbers are USA (apologies to international readers for the 800 numbers, but they're all I've got). I don't know anything about some of these companies; I suspect some of them work with HSM from other vendors rather than their own packages.
I've indicated on the list various reports of companies OEMing from each other; this is not out of disrespect for the work involved in OEMing/supporting or porting such complex software, but an attempt to divide the HSM vendors into "families" with similar capabilities (occassionally on very disparate platforms).
Vendor Product Contact ------ ------- ------- Advanced Archival Products AMASS (303)792-9700 * Advanced Software Concepts (ASC) (619)737-9544 Alphatronix ASC (919)544-0001 Artecon ASC (619)931-5500 AT&T CommVault DataMigrator (908)935-8000 Automated Network Technologies (ANT) Metior (303)789-2506 * Computer Associates International (800)225-5244 Computer Upgrade (808)874-8807 Convex Computer UniTree (214)497-3085 * COSMIC (NAStore) (706)542-3265 + Cray Research DMF (800)BUY-CRAY * Digital Equipment (DEC) NETstor (800)344-4825 Dorotech (703)478-2260 Epoch Systems (508)836-4300 * E-Systems FileServ ?* File Tek Storage Machine (301)251-0600 Fujitsu Computer Products of America OSM (408)432-6333 Hewlett-Packard OmniStorage* ,NETstor (800)637-7740x8509 HIARC (714)253-6990 IBM UniTree (800)225-5426 Introl (612)788-9391 Large Software Configurations (LSC) (612)482-4535 * Legent $OSM (703)708-3000 National Storage Lab (NSL) HPSS +* NETstor (Cheyenne) $NETstor (612)890-9367 (OpenVision UniTree (510)426-6400 *) Platinum NetArchive HSM (708)620-5000 * Qstar Technologies (301)762-9800 Raxco (301)258-2620 Software Partners/32 (508)887-6409 Storage Technology (STK, StorageTek) (303)673-5151 T-mass ? Tracer XFS ? UniTree Software UniTree (510)833-9344 * * = Info elsewhere in FAQ + = not commercial product ? = no contact info $ = original developer (no mark indicates OEM)
IBM also has HSM for MVS, called, imaginatively, HSM.
There is the storage home page. www.storage.ibm.com I
have also found references to System Managed Storage SMS and HSM and
DFHSM (Data Facility Hierarchical Storage Manager) but could find no
online information. There are probably manuals like DFHSM Version 2
Release 5.0, General Information manual (GH35-0092) if you are a real
glutton for punishment and have a friend at ibm.
So we have ADSM and DFHSM and DFSMS and probably others. But not much online information. Sorry.
A little searching from the www.ibm.com might turn up something
too.
(Del Cecchi,
A company from Atlanta, GA named Chili Pepper Software (404-339-1812) and 3M have gotten together in some fashion to make HSM software for PCs using QIC. (rdv, 94/9/5)
Runs on Netware servers. Transparent to most clients, but has a neat feature: if you use a special TSR and DLL on client PCs, when it has to retrieve a file from secondary or tertiary storage, it can give you an estimated retrieval time and the option to abort. (516)484-5110, (800)243-9462. (rdv,95/02/14)
Simple HSM for 4mm tape drives under MS-DOS. A limited freeware version is available.
More info at www.datman.com.
Voice: 708-369-7112 Fax: 708-369-7113 (Kan Yabumoto,
yabumoto@datman.com, Nov. 1995)
Try:
Avail Systems 4760 Walnut St Boulder, CO 80301 voice: +1.303.444.4018 fax: +1.303.546.4219
dave_skinner@intellistor.com (Dave Skinner) (95/2/12)
Avail's product, NetSpace HSM, has been selected by Microsoft to be
incorporated into future versions of NT, and also provides a link
between NetWare and IBM's ADSM. NetSpace also runs on Novell NetWare
systems. See www.avail.com ("Wight, Risa"
DEC's old Tops-20 OS supported offline files, and would generate an automatic request to the operator to mount a tape when the user accessed the file. When you listed a directory, it would show you which files were online and which off.
DEC's OpenVMS has some sort of support for this now. VMS 6.1 supports "shelved" files.
There is also the product Virtual Branches, from Acorn Software, which does HSM for MO and CD-ROM for OpenVMS.
Acorn Software, Inc. 267 Cox St. Hudson, MA 01749 voice: (508)568-1618 fax: (508)562-1133 Internet: info@acornsw.com
There are several packages around (mostly for PCs) that will let you use a tape drive like a disk drive. Of course, it's _very_ slow unless it uses some disk-based information as well.
See www.tapedisk.com for one such product. (rdv, 96/11/4)
email me at rdv@isi.edu
Copyright 1996 Rod Van Meter