Three criteria for an appropriate taxonomy of benchmark queries are
suggested.
- The taxonomy should be schema and instance independent. This
criterion helps ensure that the taxonomy will persist when the
benchmark database schema evolves as new versions appear. Ideally,
this will allow for an incremental mode of work, where only new
queries need to be categorized and existing queries do not need
re-categorization.
- The taxonomy should provide comprehensive coverage of
benchmark queries. Comprehensiveness is desirable to avoid holes and
point to many categories of queries.
- The taxonomy should be useful when structuring the
presentation of benchmark queries. Most importantly, it should
provide sufficient structure. Thus, taxonomies that have only few
categories and that map many queries to single categories are
problematic. If the number of categories is excessive for
presentation purposes, classes of categories may be identified with
individual sections.