Quality Characteristics for Software in HPC Environments: A Systematic Literature Review

Abstract

Research outputs in general require certain qualities to facilitate reuse as described by the FAIR Principles. For research software specifically, software engineering methods can help realize these goals. However, the desired qualities may differ between commercial and research software or even software in high-performance computing (HPC) environments. This work aims to analyze the current research on software quality in HPC and, in particular, identify important quality characteristics. Therefore, we conducted a systematic literature review on 29 relevant papers and compared our results to the established ISO/IEC 25010 SQuaRE (Systems and software Quality Requirements and Evaluation) quality model. In our analysis, the most frequently considered quality characteristics are performance, correctness, portability and maintainability, as well as trade-offs between performance and other characteristics. Compared to SQuaRE, our findings do not include compatibility, security and safety aspects. The insights from this study provide a starting point for research software engineers in quality considerations for their applications. Additionally, the comparison to SQuaRE could indicate gaps that should receive more attention to improve the reusability of HPC applications.