Return-Path:From: Hal Abbott Subject: METIOR Storage Management Software To: albertk@zeus.cs.kun.nl Date: Thu, 7 Jul 1994 10:12:04 -0600 (MDT) Cc: rdv@alumni.caltech.edu X-Mailer: ELM [version 2.4 PL21] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 10002 I saw your comments on comp.arch.storage and felt I should respond privately with information about our METIOR (TM) Software. It is currently available on Sun, SGI and Hewlett Packard 9000/700. METIOR is currently installed at Royal Melbourne Institute of Technology, and the Australian Geological Soceity Office (AGSO) in Australia. It has recently been installed at Mayo Clinic by Martin Marietta Corporation with their new CTA-2000 tape drive. Here is the information. If you wish more, you may contact me at 303.789.2506. Good Luck! rdv@alumni.caltech.edu^H METIOR (TM) METIOR is the first of a new generation of storage management software products from Automated Network Technologies (ANT). The company was founded in February, 1993 by a team of mass storage specialists who have years of field experience. METIOR is the product of that experience, including how we sell it. Our pricing model is as innovative as the software itself. With an entry-level price of $650, METIOR is affordable by all. Even site licenses and source-code licenses are available at rational prices. The team at ANT has been listening to you, and have delivered the storage management software product you have been asking for. Infinite Disk Capacity One of the most fundamental problems associated with computing is automatically solved by using METIOR. It uses removable media to create the appearance of infinite disk capacity. To accomplish this, it automatically copies files from magnetic disk to removable media. When more space is needed on the disk, new files are simply written over the space previously occupied by copied files which are no longer active. These processes are called migration (copying) and purging (writing over). Candidates for purging are based on a combination of a least- recently-used algorithm and file size. Removable media may be disk or tape and may be either manually or robotically mounted. Labeling and location of removable media is also automatically managed. Backup as a Constant Background Process Another fundamental problem which is automatically solved with METIOR, is backup. As METIOR migrates files from one hierarchy level to another, the user has the option to have METIOR simultaneously make multiple file copies. Multiple file copies are always maintained on different physical volumes, allowing at least one to be moved to an offsite location for disaster recovery. With this option, there is no need to have backup as a separate process. METIOR's backup is superior to conventional backup since the time between the creation of a file and the time when it is automatically duplicated to removable media is shorter, thus reducing exposure to data loss. This interval is selected by the user. User Restores When a file is requested, METIOR always attempts to service the request from the highest level of that fileUs hierarchy. METIOR is aware of the location of all copies of all files, but if for any reason a request fails at one level, it simply attempts the retrieve again from the next lower level until it is successful. If the reason for the failure was a disk crash, the only inconvenience to the requestor is the delay associated with mounting the removable volume which contains the file copy, and reading from it. The only involvement by a systems administrator is that associated with repairing or replacing the faulty disk. Point-and-Click Install and Operate METIOR was designed, from the beginning, to be easy to install and to use. Graphical user interfaces (GUIs) are used throughout. The design goal was to make it possible for the majority of end users to do their own installations, even with standard robotic devices, in less than one hour. Use of GUIs does not mean that flexibility must be compromised. A command line interface is also present for those who need or prefer it. METIOR's interface is unsurpassed! An Added-Dimension Evidence of the flexibility of METIOR is its ability to simultaneously manage up to seven different kinds of storage devices, plus off-line storage. Within a large networked community it is unlikely that one set of general-purpose storage devices will satisfy the needs of all. It is also unlikely that one set of rules for migration will satisfy the needs of all. With METIOR, individual users or workgroups of users may choose that subset of storage devices, whether attached or available over the network, which best satisfies their needs. Hierarchies may have the disks in the local user workstations at their top level; a robotic magneto-optical disk attached to a file server across the network as their second level; some inexpensive robotic tape, which may be attached to the same server, as their third level; and a remote disaster recovery location as their forth level. Within the same network community, another group of users uses a high-performance RAID subsystem attached to a Graphics Supercomputer as the highest level of the hierarchy. Migrating over a fiber-optic network to the fileserver, high-performance robotic tape such as the CTA-2000 is used as level two, and a remote disaster recovery location is used for the third level. Another group is similar to the one above with the exception of an additional requirement to use WORM for permanent storage. This necessitates addition of a RAID device on the fileserver to act as a buffer for speed-matching the high-performance data, which must be permanently stored on a lower-performance WORM device. Managed Storage for All Real world examples of demanding environments like the one represented in the diagram are common. Multimedia production, animation production, and medical imaging are just a few .Whether cost, performance, or permanence of media is the most important factorQMETIOR lets you make choices for the right reasons, without compromise. For the stand-alone user with a DAT or optical drive who wants the same unlimited disk capacity with automatic backup, METIOR will give it to you for only $650. Large or small, METIOR is the answer to your storage management needs. Open Systems METIOR was implemented from the proposed IEEE Mass Storage System Reference Model, Version 5. METIOR is the only commercial product currently available which uses this model for its design document. Fully Distributed Where most storage management software must run on a monolithic fileserver, METIOR may be distributed across multiple processors on the network. In high-performance environments this will enable I/O demands to be shared among multiple processors, each with a separate copy of the storage server software. RPC is used for interprocess communications, and applications programming interfaces (APIs) are available for all modules. Parallel Processes METIOR is multi-threaded to take full advantage of multiprocessors and parallel I/O. ItUs parallelism is limited only by the processor(s) upon which it is operating. Access The file system may be accessed through native NFS, FTP, or through the virtual file system interface, without conflict to non-METIOR processes. Unlimited Capacity 64 bit addressing is used throughout METIOR. This allows files as large as 16 Exabits to be stored in the file system. Domestic Prices METIOR..................................$650 10 Node Pack........................$4,500 25 Node Pack........................$9,000 50 Node Pack.......................$15,000 Site License (1st Platform)...$25,000 Site (2nd Platform)...............$10,000 Source Code (1st Platform)..$50,000 Source Code (2nd Platform).$10,000 Robotic Control Software HP 16 ..................................$3,000 HP 32 Platter.......................$5,000 HP 88 Platter.......................$9,000 HP 144 Platter....................$11,500 HP Site...............................$23,000 HP Source Code ...............$.11,500 Exabyte 10..........................$5,000 Exabyte 60........................$10,000 Exabyte 120.......................$15,000 Exabyte Site.......................$30,000 Exabyte Source Code ........$15,000 Odetics/ATL 5480/2640.....$11,500 Odetics/ATL Site................$23,000 Odetics/ATL Source Code $.11,500 Digital 7 Stacker...................$3,000 Digital Site............................$6,000 Digital Source Code .............$3,000 Metrum 48..........................$29,000 Metrum 600.........................$59,000 Metrum Site.......................$118,000 Metrum Source Code ..........$59,000 StorageTek SCSI ACS..........$29,000 StorageTek Site.....................$58,000 StorageTek Source Code ......$29,000 A registered binary license is a prerequisite for any source code license. Other restrictions apply. Multiple identical (manufacturer and capacity) robots may be attached to the same server for no additional royalty. Mixed (manufacturer or capacity) robots attached to the same server are charged as multiple units, at the prices listed above. Control software for identical robots attached to the second and subsequent different servers at the same site are priced at 50% of the first system. Integration, Installation and Training are available from ANT........................Call Automated Network Technologies 3333 South Bannock Street, Suite 945 Englewood, CO 80110 USA Phone 303.789.2506 FAX 303.789.2438 Email hal@anthill.com Authorized Resellers USA Martin Marietta Corp.,CS 1 Federal Street Camden, NJ 08102 Mr. Michael Krans 609.338.3590 AERA Corporation 54 Chestnut Hill Lane, North Williamsville, NY 14221 Mr. Gary Block 716.636.4117 South Pacific Dawn Technologies 1 Waltham Street Artarmon Sydney, NSW 0264 Australia (02) 906.1822 Europe GymData Software & DV-Beratung GmbH Uber der Hohe 16 76689 Karlsdorf-Neuthard 2 Germany Mr. Leo Kistner (0 72 51) 4 48 11 To: habbott@csn.org CC: albertk@zeus.cs.kun.nl,march@media-lab.mit.edu In-reply-to: <199407071612.AA11305@teal.csn.org> (message from Hal Abbott on Thu, 7 Jul 1994 10:12:04 -0600 (MDT)) Subject: Re: METIOR Storage Management Software --text follows this line-- Thanks for the info. I've got several questions for you: 1) What good does the basic software package do you without the robotic control software? Oh, I guess that gets you the manually mounted tape version? 2) What's your definition of "infinite disk capacity"? I've seen software packages with "infinity" being less than the capacity of an RSS-600! Coding something truly infinitely expandable generally requires lisp-like large integers, which are difficult to store in the fixed-size structures usually associated with file systems, so I suspect that by "infinite" you mean "very large" instead. Are we talking 64 bits? 128? 3) I'm very interested in hearing more about networked multiple servers, especially remote disaster recovery sites. 4) One of the examples cited mentions using local workstation disks as the top level of the hiearachy. I assume that means that the on-disk format for the disks is not UFS? How is performance impacted? 5) Speaking of performance, let's start with a configuration I know something about. Take, say, a Sparc 10 server, with a 4MB/s hard disk, and an attached Metrum drive. How fast can files be written? 6) Is there a limit on the amount of dirty cache, i.e. data stored on disk but not yet backed up to tape? 7) On a single server, can multiple tape drives be writing in parallel, or only reading? Unitree supports only one, AMASS has the restrictions/flexibility I mentioned in c.a.s, don't know about Epoch. 8) On the Sun, which operating systems are supported? 9) Do you provide a library for accessing 64-bit files, or are you dependent on the operating system for that? My impression is that true OS support for 64-bit file systems is still a ways off for most Unix systems, though I think OpenVMS and Windows NT both have it to some extent. 10) What's the background of your company's principals? Being in Colorado they could be from Storage Tek, or Metrum, or... 11) No distributor in Japan? Looking for one? The storage software market is getting crowded, but if you can follow through on some of these issues, you may solve some of the nagging problems such as disaster recovery. Good luck! --Rod Return-Path: Date: Fri, 8 Jul 1994 12:19:10 +0200 From: Albert Koppes To: rdv@alumni.caltech.edu Subject: AMASS info Thanks for the info on AMASS. Epoch have features which are equivalent to these volume groups you are describing, but without limitation possible on capacity. It is called staging trails in Epoch terminology. It is highly probable that the cache disk is the bottleneck, as we will be using a very limited capacity cache. The reason for this is that the files will arrive to the system with a regular and predictable pattern. As soon as a file is received, it will be forced to the optical disk, and not left under own Epoch control in magnetic disk. In the latter case, Epoch would decide himself on when to stage out data to optical disk, depending on the predefined watermarks setup. This would probably cause a very high amount of write every couple of hours, which would otherwise have been equally spread by the forced archiving. Here I have some questions for you on AMASS: As you seem to be running it on Sun Sparc, do you have any info on performances for read and write speed? What is the typical file size you are using? Does it support an OSF-Motif interface? What is a typical price for AMASS, including jukebox conrol software? Do you have the address of a European distributor? Frank Zeppenfeldt EUMETSAT Am Elfengrund 45 62424 Darmstadt-Eberstadt GERMANY To: habbott@csn.org CC: march@media-lab.mit.edu Subject: Metior Pricing --text follows this line-- I assume licenses are required for each client of the METIOR server? For example, if you had one server with a Metrum robot, and ten workstations using the server, you would need 11 licenses? Is this true even if the clients are diskless? Can the clients in a package be of different architectures? Can a Metior filesystem simply be mounted as an NFS partition from client machines? If so, I assume the NFS clients wouldn't need METIOR licenses. Why is the price for a Metrum RSS-48 so high? That's about half the price of a 600, even though it's only 1/12 of the capacity. I'd love it if you could send me some more detailed technical information: Rod Van Meter ASACA Corp. 3-2-28 Asahigaoka Hino-Shi, Tokyo 191 JAPAN Return-Path: From: Hal Abbott Subject: METIOR Information To: rdv@alumni.caltech.edu Date: Fri, 8 Jul 1994 11:34:21 -0600 (MDT) X-Mailer: ELM [version 2.4 PL21] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 9761 To: Rod Van Meter Date: Fri, 08 Jul 1994 04:03:29 -0700 Subject: Metior Pricing To: habbott@csn.org Cc: march@MEDIA-LAB.MEDIA.MIT.EDU X-Envelope-To: habbott@csn.org I assume licenses are required for each client of the METIOR server? For example, if you had one server with a Metrum robot, and ten workstations using the server, you would need 11 licenses? Is this true even if the clients are diskless? Can the clients in a package be of different architectures? ANSWER--Normally, client licenses are required only where METIOR is managing the local disk. On a diskless client the server filesystem would normally be NFS mounted, or files could be transferred utilizing FTP. I can think of some unusual circumstances where a diskless client may be part of a data acquisition system or a distributed system with network attached peripherals where this would not be true. If the METIOR filesystem on these devices is being used on the client, through the virtual file system interface, then a client license would be required. ANSWER--The clients may be mixed. We currently support SunOS 4.1.3, SGI/IRIX 5.2(beta), and HP/UX 9.x. Solaris 2.3 will be in beta at the end of July or early August. (We have a customer waiting). Windows NT will be available before the end of the year. (More customers waiting). Can a Metior filesystem simply be mounted as an NFS partition from client machines? If so, I assume the NFS clients wouldn't need METIOR licenses. ANSWER--That is correct. Why is the price for a Metrum RSS-48 so high? That's about half the price of a 600, even though it's only 1/12 of the capacity. ANSWER--We look at the price from a different perspective. >From our point of view, the 600 is aggressively priced. I'd love it if you could send me some more detailed technical information: ANSWER--It is on the way. Date: Thu, 07 Jul 1994 17:48:01 -0700 Subject: one more... To: habbott@csn.org X-Envelope-To: habbott@csn.org I'm making some additions to the comp.arch.storage FAQ that I hope will appear in a future version. Is it okay if I use some of your information and list you as a contact for Metior? --Rod ANSWER--Please do. Thank you. Date: Thu, 07 Jul 1994 17:10:39 -0700 Subject: Re: METIOR Storage Management Software To: habbott@csn.org Cc: albertk@zeus.cs.kun.nl, march@MEDIA-LAB.MEDIA.MIT.EDU X-Envelope-To: habbott@csn.org Thanks for the info. I've got several questions for you: 1) What good does the basic software package do you without the robotic control software? Oh, I guess that gets you the manually mounted tape version? ANSWER--Exactly. It could be manual optical as well. We believe one of the major reasons HSM has not flourished in the mainstream market has been that the entry-level price was a major obstacle. There was no way for customers to try the software without making a major dollar commitment. That is no longer true with METIOR. We also believe that the Windows NT/Chicago/Cairo will open a new market, even to laptop computing where the problem is that when files are moved to removable media, they disappear from the directory--and become lost. With METIOR on the laptop all files remain listed in the directory, and the removable media location (which PMCIA?) is managed automatically. 2) What's your definition of "infinite disk capacity"? I've seen software packages with "infinity" being less than the capacity of an RSS-600! Coding something truly infinitely expandable generally requires lisp-like large integers, which are difficult to store in the fixed-size structures usually associated with file systems, so I suspect that by "infinite" you mean "very large" instead. Are we talking 64 bits? 128? ANSWER--64-bits. At one gigabyte per second, it would take more than sixty-three years to fill the filesystem. I am fifty-seven years old, and will be gone before the filesystem is filled. That makes it unlimited capacity to me. 3) I'm very interested in hearing more about networked multiple servers, especially remote disaster recovery sites. ANSWER--METIOR was truly designed from the proposed IEEE model, version 4. When version 5 was made available, we incorporated the changes. Therefore, it is designed to allow all modules to be distributed. Today, bitfile clients and storage servers may be freely distributed across the network to allow I/O demands to be distributed across multiple processors/backplanes within a network community. This is also how network attached peripherals are implemented. Unfortunately only the NSL/IBM/Maximum Strategy RAID controller, which is a closed device, is available for this purpose. Using RPC, a layer in any storage hierarchy may be a remote site, connected through standard links like T1 or SONET. With METIOR, multiple copies of subsets or all files may be made at any point in migration. Multiple copies are always on different physical volumes, which may be automatically accessible as exported volumes, through the remote links. 4) One of the examples cited mentions using local workstation disks as the top level of the hierarchy. I assume that means that the on-disk format for the disks is not UFS? How is performance impacted? ANSWER--We are concentrating our efforts in this area today. Currently, performance does not meet our own standards, but we believe that we will be able to match the performance of UFS within ninety days. We have achieved that today on the server, including things like ls-l. 5) Speaking of performance, let's start with a configuration I know something about. Take, say, a Sparc 10 server, with a 4MB/s hard disk, and an attached Metrum drive. How fast can files be written? ANSWER--It has been several months since we have had a 2150 to test, so I cannot give you observed performance for the 2150. However, in another life, our development team was the first to drive the 2150 at full bandwidth using UniTree on SGI. These systems are installed at Martin Marietta Corp. and at NASA Goddard. METIOR is faster than UniTree. ANT is currently under contract to Martin Marietta Corporation for the purpose of integrating METIOR with their CTA-2000 tape drive. This system is currently faster than the 2150 when running on a Sparc 10, and will ultimately achieve sustained write rates of about ten MBytes/sec. I believe this system will be demonstrated at the supercomputer show this year. 6) Is there a limit on the amount of dirty cache, i.e. data stored on disk but not yet backed up to tape? ANSWER--No. Unlike our competitors we have no two-gigabyte limitations. 7) On a single server, can multiple tape drives be writing in parallel, or only reading? Unitree supports only one, AMASS has the restrictions/flexibility I mentioned in c.a.s, don't know about Epoch. ANSWER--METIOR has no practical limitations on its parallelism, reading or writing, in any combination. We are solely limited by the ability of the hardware. 8) On the Sun, which operating systems are supported? ANSWER--Today, SunOS 4.1.3. By the end of this month, we will be in beta on Solaris 2.3. 9) Do you provide a library for accessing 64-bit files, or are you dependent on the operating system for that? My impression is that true OS support for 64-bit file systems is still a ways off for most Unix systems, though I think OpenVMS and Windows NT both have it to some extent. ANSWER--METIOR uses our own 64-bit library routines. Along these lines we offer a utility we call the * Recognizor * which allow us to manage legacy filesystems without conversion of the original physical volumes. We are currently using this utility at Mayo Clinic to manage large amount of tapes which were written in ArFS. The Recognizor scans the physical volumes and the current management system for the metadata and incorporates it into the METIOR database. from that point forward, when any file is requested, it is serviced by METIOR. With the Recognizor, we are able to replace existing Epoch, AMASS, and UniTree systems with minimal disruption to normal operations. A future feature-high priority-will be to enable a layer in the hierarchy, with any device, which will write standard UNIX or Windows NT filesystems, for export purposes. That layer may also be used to easily replace METIOR when it becomes the obsolete legacy system. 10) What's the background of your company's principals? Being in Colorado they could be from Storage Tek, or Metrum, or... ANSWER--None of the above. We all come from large scale, high-performance computing and integration companies. We converged in Denver with Titan Corporation for about one year. At Titan, we put their storage business unit in motion with UniTree, but discovered that, like most defense companies, they had no understanding of the dynamics necessary to build a commercial business. In January, 1993 we departed and founded ANT. 11) No distributor in Japan? Looking for one? ANSWER--Absolutely. After you have received and digested the material I have mailed today and still have interest, please contact me. The storage software market is getting crowded, but if you can follow through on some of these issues, you may solve some of the nagging problems such as disaster recovery. Good luck! --Rod ANSWER--Thanks for your interest. We are in the process of building some strong relationships which we believe will take us to our ultimate goal of being a major supplier of integrated mass storage systems. Our commitment and belief in METIOR is so strong that we have personally bet our homes on it. Once again, Thanks for your interest. All the ANTs look forward to hearing from you. ------Hal P.S. Did you know that ants never sleep? Return-Path: From: Hal Abbott Subject: Re: METIOR Information To: rdv@alumni.caltech.edu (Rodney D. Van Meter) Date: Fri, 8 Jul 1994 21:05:29 -0600 (MDT) In-Reply-To: <199407090233.TAA05361@alumni.caltech.edu> from "Rodney D. Van Met\ er" at Jul 8, 94 07:33:14 pm X-Mailer: ELM [version 2.4 PL21] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 240 METIOR is pronounced like meteor, but the meaning is different. METIOR is latin\ for "...to set the standard for." It is also the root of metrology. The package is on its way to you. I'm looking forward to your reaction. Best regards--Hal