The Impact of GitHub Copilot on Test-First Development (2024.09-2024.12) [paper] [code]
In a class project, we examined the impact of GitHub Copilot on Test-First Development, a software development paradigm where tests are written before implementation. We conducted a between-subjects (no Copilot vs Copilot) pilot study, consisting of coding tasks and pre- and post-task surveys. Participants iterated between writing a comprehensive API and test suite for a problem, with or without Copilot. We found that while Copilot enhanced coding speed, it resulted in superficial problem comprehension and decreased scope of the test suites.