Subject: TSQL Benchmark STATUS

From: "Christian S. Jensen" <[email-masked]>
Date: Sun, 6 Jun 1993 23:25:41 +0200

**********************************************************************
**                      The TSQL Benchmark                          **
**              Task 4--Benchmark Queries --- STATUS                **
**********************************************************************

Dear colleague,

The preparation of the first version of the TSQL Benchmark was divided
into four tasks. The first three tasks---deciding on the benchmark
database schema and instance, and the taxonomy of benchmark
queries---have been accomplished. The last task, that of populating
the taxonomy with queries, is close to completion. Specifically, there
were thirty classes, and five queries were to be supplied for each
class. At this time, at total of twenty classes have been chosen by
volunteers, and only the following ten classes are still available for
contributors.

                           Classes Available
                           *****************
	O1.S7 		O1.S8		O1.S9		O1.S10

	O2.S4		O2.S5		O2.S6		O2.S10
	
	O3.S6		O3.S8

While time is short, contributors are still encouraged to choose one
or two classes (i.e., write 5 or 10 queries). All that do so will
become coauthors of the paper. If you are interested, but have
questions and need more information, please feel free to contact me
([email-masked]).

Immediately following is a Latex document that may be used when
viewing queries. The document also provides additional explanation.
Note that this additional explanation is added here to explain the
categories and classes. Such explanation is not required in the
submitted queries.

Finally, contributors were also encouraged to write appendices with
comments and observations that may be useful in the continued work
with the TSQL benchmark. The firm deadline for receipt of such
appendices is noon on Wednesday, June 9.

Best regards,
Christian S. Jensen, editor
Aalborg University
[email-masked]

\documentstyle[11pt]{article}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Latex template for queries
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\addtolength{\textwidth}{1.485in} \setlength{\oddsidemargin}{.1in}
\setlength{\evensidemargin}{.1in} \addtolength{\topmargin}{-.85in}
\addtolength{\textheight}{1.8in}

\long\def\comment#1{}

\long\def\query#1#2#3{\begin{verse} {\bf Query \no} {#1} \end{verse}
\begin{verse} {\bf Answer:} {#2} \end{verse} \begin{verse} {\bf
Category:} {#3} \end{verse}}

\newcounter{qnumber}
\newcounter{rnumber}[subsection]
\newcounter{gnumber}[subsubsection]
\newcommand{\no}{\setcounter{qnumber}{\value{subsection}}
\setcounter{rnumber}{\value{subsubsection}}\protect\refstepcounter{gnumber}
Q \protect\theqnumber.\protect\thernumber.\protect\thegnumber:}

\begin{document}

\subsection{Example With Explanations}

\subsubsection{Class O1.S3 (Duration, Element, Computed)}

\query{Find the names of employees that have been in a department
  named Toy for a shorter period than has DI.} 
  {``Ed'' and ``Edward.''}
  {(Projected, None) / (Duration, Element, Computed) / (=, Constant)
    (=, Constant)}

  \noindent {\it The employee ED has been in a department named Toy
    for a period which is shorter than that of DI. The categorization
    is with respect to Figure 6. ``(Projected, None)'' indicates that
    only part of the attributes of the argument relation are present
    in the result and that there is no valid-time component. Next,
    ``(Duration, Element, Computed)'' indicates that a duration based
    predicates is used on element-valued arguments which are both
    derived from the valid-times of stored facts.  Finally, ``(=,
    Constant) (=, Constant)'' indicates that there are two
    non-temporal selection predicates that test for equality of an
    attribute value with a constant (i.e., the person must be DI and
    the department must have name Toy).}

\query{Find the current name and department name for the persons which
  made \$40K for a longer period than DI did.}
  {``(Edward, Book).''}
  {(Projected, None) / (Duration, Element, Computed) (Containment,
    Event, Explicit) / (=, Constant)}

  \noindent {\it In this query, there are two valid-time selection
    based predicates.  The one used for categorization compares the
    duration of time when a person makes \$40K with the period of time
    that DI makes \$40K. The other selects the name and department
    that overlap with the current time of qualifying persons.}

\end{document}

---

From: [email-masked] (John Roddick)
Date: 07 Jun 1993 09:36:54 +0930

>                           Classes Available
         ^^^^^           ^^^^^
>        O3.S6           O3.S8
                         ^^^^^
I'll have a go at the three marked above.  

Question - would it be useful if we had a list of who's opted for the other
queries as it might make sense to liaise with those doing the associated
queries?  Then again, maybe if we came up with the queries blind they would
be more diverse / interesting / complete.  Any comments?

John.

---------------------------------------------+--------------------------
John Roddick                                 | Rm.  MC3-14, Levels
Data and Software Engineering Research Group | Ph.  (08) 302 3463
School of Computer and Information Science   | Int. +61 8 302 3463
Faculty of Applied Science and Technology    | Fax  (08) 302 3381
University of South Australia                | Int. +61 8 302 3381
The Levels Campus                            | Net. [email-masked]
SOUTH AUSTRALIA     5095

---

From: "Rick Snodgrass" <rts>
Date: Mon, 7 Jun 1993 09:59:34 MST

A few minor points.

1. There are seven classes still available, and almost exactly that
many authors who have not volunteered to handle a class or two. If
you are still "class-less" (sorry for the pun...), please grab at
least one class. Five queries can't be that hard to write.

2. Christian stated that the deadline for receipt of queries
was last Friday (I think). If you volunteered, but haven't yet written your
queries, please do so by the firm deadline stated in his last message
of noon, Wednesday, June 9. (Since Christian didn't indicate the
time zone, let's be most liberal, and use PDT, which translates,
I think, to 9pm Danish time.)

3. The current draft of the taxonomy is 9 pages. Of that material,
Christian wrote at least 7 pages. Also, he had worked exceedingly hard
at organizing this effort. I recommend that his name be moved to the
front of the author list, to acknowledge that he has done more than anyone
else in writing this benchmark. All other names should remain in
alphabetical order, unless one or more individuals clearly contribute
significantly more than the rest of us.

---

From: [email-masked] (Alexander Tuzhilin)
Date: Mon, 7 Jun 93 15:58:14 EDT

The latest messages from Christian and Rick concerning the TSQL
Benchmark Queries have justifiably asked those who have not yet
contributed a set of queries to the benchmark to please do so by the
indicated deadline.  We sat down together this morning to do just
that, and in trying to fully understand the categories we got to
reflecting again upon the purpose of the benchmark as a whole.
Basically we ran up against the same discomfort with this benchmarking
effort that we indicated in an earlier message in this forum.  We
would like to restate our reasons for this discomfort.

We do not understand the claim in the Draft document that the
benchmark will contribute to evaluating the "user-friendliness" of
temporal query language proposals. User-friendliness is not
typically measured in this way.

It still seems to us that the whole idea behind this "semantic
benchmark" is to provide some measure of what should be expressible in
a temporal query language.  As we indicated in our earlier message, we
believe that a list of queries is not the appropriate way to provide
such a measure.  One of the criteria mentioned for the taxonomy (p.5)
is that it should provide "comprehensive coverage of benchmark
queries."  We do not feel that a classification that does not have
some sound theoretical basis can be considered comprehensive.  For
example, in the discussion of the Valid-Time Selection taxonomy (p.7,
and Figure 4), "comparison operators" are mentioned, yet only a few
examples (spanExceeds, preceded, meets, etc.) are provided. What is
the complete set of these operators?  In temporal logics there are
dozens of different operators that have been defined (since, until,
before, after, while, always, sometimes, etc.), and different subsets
of these operators, combined with different underlying models of time,
have provably different expressive powers.  Without some clear
theoretical guidance, there is no basis for the selection of the 30
categories described in the benchmark document, nor any basis for
formulating "five example queries" in any meaningful way.

We don't want to sound too negative in this message, but we would like
to encourage everyone to think again about the purpose of this
benchmark, the issue of the "comprehensiveness" of the benchmark, and
the issue of the expressive power of temporal query languages. We hope this
message will be constructive, and will generate discussion and comments.
We welcome both.

Alex Tuzhilin and Jim Clifford

---

From: "Rick Snodgrass" <rts>
Date: Mon, 7 Jun 1993 13:41:24 MST

Expressiveness and user friendliness are two separate concepts.
Contrast SQL and the relational algebra. The former is generally judged
to have greater user friendliness, whereas both can be shown to have
equivalent expressive power (with appropriate provisos). In the papers
I have read, user friendliness has been argued via examples (c.f.,
the current benchmark), or with psychological testing via user
studies (again, with examples).

The first page of the benchmark states "The benchmark is ...not
a substitute for a rigorous theoretical study of expressive powers
of various temporal query languages." Hence, it explicitly removes
expressive power from those issues it considers further.

I guess we should consult the originators of the term "semantic
benchmark," Ed and Patrick, to hear whether they agree with the comment
that "the whole idea behind this 'semantic benchmark' is to provide
some [presumably sound theoretical basis for a] measure of what should
be expressible in a temporal query language."