Evaluation measures for ontology matchers in supervised matching scenarios



Item Type: Dataset
Title: Evaluation measures for ontology matchers in supervised matching scenarios
Date: 7 May 2013
Creator : Ritze, Dominique, Paulheim, Heiko and Eckert, Kai
Divisions: School of Business Informatics and Mathematics > Wissensrepräsentation u. Wissensmanagement (Juniorprofessur) (Stuckenschmidt 2005-2008)
DDC
Classification:		 004 Computer science, internet
Abstract: Precision and Recall, as well as their combination in terms of FMeasure,are widely used measures in computer science and generally used to evaluate the overall performance of ontology matchers in fully automatic, unsupervised scenarios. In this paper, we investigate the case of supervised matching,where automatically created ontology alignments are verified by an expert. We motivate and describe this use case and its characteristics and discuss why traditional, F-measure based evaluation measures are not suitable to choose the best matching system for this task. Therefore, we investigate several alternative evaluation measures and propose the use of Precision@N curves as a means to assess different matching systems for supervised matching. We compare the ranking of ontology matchers from the last OAEI campaign using Precision@N curves to the traditional F-measure based ranking, and discuss means to combine matchers in a way that optimizes the user support in supervised ontology matching.
URL: https://madata.bib.uni-mannheim.de/23/
DOI: https://doi.org/10.7801/23
Availability (Controlled): Download
Publication(s)
(MADOC) :		 Ritze Dominique und Paulheim Heiko und Eckert Kai (2013), Evaluation measures for ontology matchers in supervised matching scenarios

Metadata export

FileFilename / InfosLink
[img] Archive (Contains all R-scripts to plot the curves and compute the AUC. Additionally, all data we used for the plotting and computation is already included in the archives)
Filename: RScriptsData.zip


Download (5MB)
[img] Archive (Source code of the Java application contained in Application.zip.)
Filename: src.zip


Download (17kB)
[img] Archive (Application to compute the measures and generate files which can be used in combination with the R-scripts to plot the curves and compute the AUC.)
Filename: Application.zip


Download (12MB)
[img] Text (Further description of the files and instructions e.g. how to start the Java-application.)
Filename: README.txt


Download (1kB)
Depositing User: Dominique Ritze
Date Deposited: 08 May 2013 06:06
Last Modified: 02 Jul 2015 14:30

Actions (login required)

View Item View Item
Mannheim Research Data is powered by EPrints 3 which is developed by the School of Electronics and Computer Science at the University of Southampton. More information and software credits.