Researchers Use Developer Biometrics to Predict Code Quality
Informatics researchers from the University of Zurich have developed a not at all sinister-sounding system capable of predicting the quality of code produced by developers based on their biometric data. By using heart rate information, for example, they were able to quantify the difficulty a given programmer had in producing a piece of software. This information could then be used to identify likely sections of bad code. Pre-crime for software debugging, in other words.
The Zurich researchers, Sebastian C Müller and Thomas Fritz, describe their work in a paper presented this week at the 38th International Conference on Software Engineering in Austin.
Here's the not at all sinister way that Müller and Fritze open their paper: "A commonly accepted principle in software evolution is that delaying software quality concerns, such as defects or poor understandability of the code, increases the cost of fixing them. Ward Cunningham even went as far as stating that 'every minute spent on not-quite-right code counts as interest on that debt.'"
A common first line of defense against bugs and just poor quality code is the code review, the duo notes. Basically, one developer writes some code and then passes it on to someone else, who sort of acts like a code editor. The reviewer/editor looks over the completed code for defects and places where the code might be improved.
This is a costly system, however: Code reviews take time and people. Automated review systems exist and sort of work, but they run up against two barriers. "First, they are predominantly based on metrics, such as code churn or module size, that can only be collected after a code change is completed and often require access to further information, such as the history of the code," the paper notes. "Second, they do not take the individual differences between developers comprehending code into account, such as the ones that exist between novices and experts."
Enter biometrics. By looking at the programmer as they program, rather than the code after the programmer is done writing it, the system described by the Zurich researchers finds code quality issues as the code is being produced. Previous research has already show that heart rate variability (HRV) or electrodermal activity (EDA) can be accurate indicators of task difficulty or difficulty in comprehending code snippets. The more difficult the task, the higher the cognitive load and chances for errors to be made.
They tested this out with two teams of developers from Canada and Switzerland. Biometric data was collected as they wrote software and this was correlated with interviews with the developers and human-based code reviews.
The study's findings:
Amongst other results, our study shows that biometrics outperform more traditional metrics and a naive classifier in predicting a developer's perceived difficulty of code elements while working on these elements. Our analysis also shows that code elements that are perceived more difficult by developers also end up having more quality concerns found in peer code reviews, which supports our initial assumption. In addition, the results show that biometrics helped to automatically detect 50 percent of the bugs found in code reviews and outperformed traditional metrics in predicting all quality concerns found in code reviews.
The paper at least notes (barely) the obvious privacy concerns with this, but it doesn't go much further than that. It's more than just a matter of privacy, really, but of general invasiveness (which is a bit different) and the really glaring, obvious problem of monitoring programmer stress as they program itself sounding like the most stressful scenario ever—like a coding interview that never ends where you also happen to be naked. Like that.
Bleak, anxious times ahead.