Improving Software Testing Education through Lightweight Explicit Testing Strategies and Feedback
Date:
Abstract: This project aims to serve the national interest by designing tools and pedagogy that will help students learn how to write better software tests. Software testing is a critical skill for computer science graduates when they enter the workforce. A software professional with better software testing skills is likely to produce more secure, more robust, and less error-prone software. Writing effective software tests is an open-ended problem that students find very challenging. The use of checklists has been shown to be an effective practice for helping students learn other software engineering skills. This project will investigate the use of checklists to provide a procedure that students can use to write software tests. Students will learn how to write software tests and use them to evaluate the quality of software they develop. The project team will develop a software tool that will run students? tests on software and provide real-time hints and feedback on the quality of their tests. The project will also examine the benefits of testing education for students in terms of their learning outcomes and code quality.
The goal of this project is to help students develop better software testing skills so that they will be successful in the computing workforce. This project will develop new data-driven methods for generating automated testing feedback and hints for software tests written by students. The project team will conduct a longitudinal study to assess the impact of using checklists and automated feedback on students? ability to write tests using six evaluations over three years. The project will address three research questions: (1) To what extent does the checklist intervention improve students? software testing outcomes? (2) What are the benefits of automated feedback on test coverage and quality as compared to less rigorous interventions? (3) What is the impact of writing high-quality tests on student learning outcomes? The impact of the interventions will be measured using test quality, implementation quality, improvement between pre- and post-test quiz scores, self-assessment of testing ability using a survey, and the time students spend working on each phase of their lab and homework assignments. The project has the potential to have a lasting impact on how undergraduate students learn software testing. The results of this project will be disseminated through conferences and journals that focus on computer science education. The project team will post project results including tools, experiments, and educational materials on GitHub. The NSF IUSE: EHR Program supports research and development projects to improve the effectiveness of STEM education for all students. Through the Engaged Student Learning track, the program supports the creation, exploration, and implementation of promising practices and tools.
This award reflects NSF’s statutory mission and has been deemed worthy of support through evaluation using the Foundation’s intellectual merit and broader impacts review criteria.