Runtime Performances Benchmark for Knowledge Graph Embedding Methods

원문 초록

This paper wants to focus on providing a characterization of the runtime performances of state-of-the-art implementations of KGE alghoritms, in terms of memory footprint and execution time. Despite the rapidly growing interest in KGE methods, so far little attention has been devoted to their comparison and evaluation; in particular, previous work mainly focused on performance in terms of accuracy in specific tasks, such as link prediction. To this extent, a framework is proposed for evaluating available KGE implementations against graphs with different properties, with a particular focus on the effectiveness of the adopted optimization strategies. Graphs and models have been trained leveraging different architectures, in order to enlighten features and properties of both models and the architectures they have been trained on. Some results enlightened with experiments in this document are the fact that multithreading is efficient, but benefit deacreases as the number of threads grows in case of CPU. GPU proves to be the best architecture for the given task, even if CPU with some vectorized instructions still behaves well. Finally, RAM utilization for the loading of the graph never changes between different architectures and depends only on the type of graph, not on the model.