Multidimensional Data-driven Ontology Evaluation
This thesis explores the multiple dimensions to data-driven ontology evaluation. Theoretically and empirically it suggests two ontology evaluation metrics - temporal bias and category bias, as well as an evaluation approach that are geared towards accounting for bias in data-driven ontology evaluation. The topic of ontologies and their development is one that has received wide interest from both academic and industrial domains. The direct results of such a wide interest has been a plethora of ontologies on the web. The success of the semantic web has benefited from this increase in the availability of ontologies. On the other hand, evaluation of these ontologies has proved to be a challenge. From a reuse point of view, the goal has been to sift through related ontologies and decide on which one to reuse. To that end a varied number of approaches to ontology evaluation have been proposed. This thesis focuses on data-driven ontology evaluation, a subset of ontology evaluation that considers knowledge about the domain to evaluate ontologies. While ontologies are a shared knowledge base, they are also created on a specific environmental setting, time, and largely based on the modeller's perception of the domain. Moreover, domain knowledge from which they are based on is non-static and changes over different dimensions. These are the notions that have been overlooked in current research on data-driven ontology evaluation. The ultimate goal is to answer the question: How do the domain knowledge dimensions affect the results of data-driven ontology evaluation? Consequently, the thesis presents a theoretical framework as well as two metrics that account for bias along the dimensions of domain knowledge. Based on the statistical experimentation and evaluation we can conclude that the domain dimensions do bias the evaluation of ontologies. This is especially true in case of the temporal dimension where the ontologies under investigation had shown statistically significant difference in their coverage of the domain at different time intervals.