Premium
Introducing orbis: An extendable evaluation pipeline for named entity linking performance drill‐down analyses
Author(s) -
Odoni Fabian,
Braşoveanu Adrian M. P.,
Kuntschik Philipp,
Weichselbraun Albert
Publication year - 2019
Publication title -
proceedings of the association for information science and technology
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.193
H-Index - 14
ISSN - 2373-9231
DOI - 10.1002/pra2.49
Subject(s) - pipeline (software) , benchmarking , computer science , annotation , context (archaeology) , precision and recall , drill , information retrieval , measure (data warehouse) , natural language processing , data mining , artificial intelligence , engineering , programming language , archaeology , geography , mechanical engineering , marketing , business
Most current evaluation tools are focused solely on benchmarking and comparative evaluations thus only provide aggregated statistics such as precision, recall and F1‐measure to assess overall system performance. They do not offer comprehensive analyses up to the level of individual annotations. This paper introduces Orbis, an extendable evaluation pipeline framework developed to allow visual drill‐down analyses of individual entities, computed by annotation services, in the context of the text they appear in, in reference to the entities specified in the gold standard.