Comments on the Benchmark
Subsections
Christian S. Jensen
It should be stated clearly in the scope of a benchmark what types of data models, it is relevant to.
Consensus benchmarks are useful for avoiding issues related to conflicts of interest.
Benchmarks made by users and reflecting specific applications complement generic benchmarks made by researchers.
The specificity of the benchmark database schema must be chosen carefully.
A taxonomy of queries should help structure the presentation of benchmark queries and should help ensure a complete coverage of possibilities by suggesting a wide range of categories of queries.
There is a mismatch in precision between a taxonomy based on a formal query language and queries formulated in natural language.
A detailed, covering, easy-to-use, semantic taxonomy is desirable.