Visual regression testing for information visualizations (2018)
Testing is a crucial step in software development. Because of this, many testing techniques have been developed. One of the reasons for this plurality is that different software development areas have specific needs. Therefore, there are specific testing techniques for mobile development, security aspects, and cloud computing. However, automated software testing for data visualizations is mostly uncharted territory, that has left developers testing mostly manually. Some techniques such as visual regression and GUI testing have made some advances that could help visualization developers, but they only partially address the needs of visualization testing, leaving out concepts such as testing complex interactions, animations and data validity. Moreover, current visual regression tools are so complicated that visualization developers usually ignore them altogether. This thesis contributes to this field by first exploring and mapping the landscape of automated software testing from a data visualization testing perspective, and then proposing a testing framework that focuses on the specific needs of data visualization. This research makes two main contributions: First, the exploration and mapping of the state of the art in software testing, which resulted on the validation of a need for specific data visualization testing and a proposal to improve the CMU Software Engineering Institute-s software testing taxonomy to acknowledge this need. Second, a framework that provides a pipeline to test visualizations using visual regression testing, and the development of a prototype open source library that helps as a proof of concept of the framework. This framework proposes a novel data visualization testing vocabulary...