Randomized, Structured, Auto-graded Homework: Design Philosophy and Engineering Examples

Engineering homeworks encourage students to practice skills and apply concepts. Such homeworks are critical to a student's learning of course content and performance on high-stakes exams. Research has examined approaches to improve effectiveness of homeworks, including auto-grading for faster feedback and adaptivity to personalize a student's learning. Over the last 8 years, we have developed a homework activity framework that has been applied to multiple engineering and math disciplines with wide-spread adoptions: 600,000 students across 800 universities have submitted 90 million times. Our homework activities are integrated into web-based interactive textbooks. Such a homework activity is a sequence of progressively more difficult levels. A student must complete the first level's question to move on to the second level's question, and so on. Each level contains numerous same-difficulty questions, one of which is randomly selected when the student arrives at a level. A student's submission is auto-graded, and the student receives specific and immediate feedback to the given question and their submission. If the student answered incorrectly, then the student can try again on a new randomly-generated question of the same difficulty. Our homework activity philosophy is: (1) randomized - each question is randomly generated to enable students plenty of practice and enable instructors to reuse the activity for an exam, (2) structured - an activity is a sequence of incrementally harder questions so a student can demonstrate mastery, (3) auto-graded - a student's submission is immediately assessed and the student is provided relevant feedback. This paper describes our homework activity philosophy, including pedagogical considerations made in designing such activities, many examples across different subjects, and reasons for implementing such a homework activity. Student submission data shows that on average across all the subjects discussed in this paper, an average of 98.4% of students were able to successfully complete an attempted level.