Examining the necessity of problem diagrams using MOOC AB experiments

Abstract

Earlier research on problem solving suggested that including a diagram in a physics problem brings little, if any, benefit to students’ problem solving success. In 6 AB experiments conducted in our MOOC, we tested the usefulness of problem diagram on 12 different physics problems, collecting over 8000 student responses in total. We found that including a problem diagram that contains no additional information very slightly improves the first attempt correct rate. On the other hand, in half of the cases, removing the diagram significantly increased the fraction of students who elected to draw their own diagram during problem solving. The results suggest that in contrast to conventional wisdom, the benefit of including a problem diagram rarely justifies the cost of creating one. © 2016 International Educational Data Mining Society. All rights reserved.