My comments in TidBITS-341 about building a database into the system for applications to use hit a chord with readers. A number of people wrote in with additional comments and ideas.
Alex Sirota <adapple@apple.com> offers an additional requirement to my suggestions for what features are essential for a relational database system:
Another requirement that you didn't mention is a structured language with which to manipulate the contents of the database system. The power of an RDBMS (Relational Database Management System) over something like FileMaker Pro is the ability to manipulate the data, stored in tables, using a standard, accepted language such as SQL (Structured Query Language) and program the database procedurally with extensions to SQL that many RDBMSs support.
Shay Telfer <shay@cs.uwa.edu.au> comments:
The Newton uses a system-wide database that is accessible to any package running on it. Different applications can tightly integrate with your names, addresses, and appointments databases (known as "soups"), and can search or add information to each of these soups as appropriate. The database contains everything on the Newton, including code, graphics, etc., and it copes with people removing storage cards at will (even while the database is being searched). Such integration is extremely powerful and I think Apple should be aiming for this level of integration in the Mac OS (and I think it's what the people at Be are hoping for in the BeOS). The Newton is definitely a technology that was, and still is, ahead of its time.
Apple's Dylan implementation (may it rest in peace) was based around a database system where all the source code for each project was kept in a database. This made project revision control much easier, and was more sensible than the current way of thinking where functions are lumped together in a group of flat files on disk, which give little indication how the different parts of the software relate to each other. As far as I know, there are very few development environments out there which do this, other than Smalltalk, Prograph, and Frontier.
David Charlesworth <davec@andyne.com> writes:
I wanted to mention a few of things about using databases for storage at a system level:
<http://dev.info.apple.com/du/intro_to_opendoc/ iod4_underlying_technology.html>
An additional benefit of a public database interface is that it provides a choice of tools to query and report on the data, repair problems, and extend the data. For example, if one of the PIM (personal information manager) producers used a standard database, then you could add additional tables and fields, and use "the right tool" (whatever that may be for you) to build reports, and still have the benefit of the user interface that the PIM provides. Just think how nice it would be if your PIM and your email program and your fax software shared the same database!
There's the JET Engine shipped as part of Windows (ODBC interface, reasonable speed, full relational capability). When we wanted to build a prototype for a potential cross platform product recently, we built it on Windows because we could use the JET Engine for storage. This speeded our prototyping cycle, enabled us to ship to evaluators without licensing hassles, and enabled us to substitute other SQL-compliant databases for multi-user data sharing down the road. All of these are big wins. We spent zero time on reporting tools for it, zero time on storage-related optimizations, and had revision cycles of a few hours because we didn't need to rebuild our test data.
Marty Wachter <mwachter@ubmail.ubalt.edu> adds:
You should note that AOCE (more commonly known as PowerTalk) contained an OS object database for storing anything, however it was not widely known, and there were only low level routines documented to deal with it under the PowerTalk API. I think the developer API for the Catalog Manager (or whatever they wound up calling the thing) contains all of the info. The actual API was really ugly, and thus no one used it - even those of us who found it.
Lawrence Conroy <lwc@roke.co.uk> writes:
As it happens, we are exploring system level databases for a project we're doing for a customer. I wholeheartedly agree with you that databases will be of increasing importance.
For instance, much as I say it through gritted teeth, Microsoft has a system level database of sorts in the Windows 95 Registry. It's rather like a collective preferences folder, so that applications can store their configurations setting there, rather than having a large number of application-specific .INI files. There is a "standard" way to access these database entries, and there's an included program (RegEdit) so users can browse and edit the entries.
There is another area where databases in a commonly understood format will be useful: databases that are accessible remotely. Related work is being started in the IETF (Internet Engineering Task Force). For example, there's a group based at Carnegie Mellon University developing proposals for a protocol for access to a remote configuration database; other groups are also working on this kind of problem, notably in the Access, Searching and Indexing of Directories (ASID) group of the IETF.
<http://andrew2.andrew.cmu.edu/cyrus/acap/>
<http://www.ietf.org/html.charters/asid- charter.html>
Microsoft has been active in the ASID group, proposing that the existing LDAP (Lightweight Directory Access Protocol) work be extended for dynamic databases using their User Location Service (ULS). This is, basically, an enhancement of their existing WINS directory that's at the heart of Windows Networks. Their Internet Draft is at:
<ftp://ftp.ietf.org/internet-drafts/draft-ietf- asid-ldapv3ext-00.txt>
In an attempt to (among other things) reinvent the telephone network, a number of directory companies are setting up multi-user remote databases that show (a) whether a user has a particular program (like a certain flavor of Internet phone application), and (b) whether or not the user has the program running at the time of the query. This kind of generally-accessible database is necessary if such applications are going to find widespread use. Otherwise it's difficult to know whether or not you can talk to someone (at all), since different Internet phone programs usually don't talk to one another. Likewise, without a way of registering the running program at startup, it's difficult for callers to know whether or not the delay in response is due to their call request disappearing into the bit bucket, or because their intended correspondent isn't running the program at that time.