Effectiveness of Rutgers Calculus Formats

During the late 1990's many Universities, including Rutgers, introduced new formats for Calculus courses. Anecdotes abound to the effect that these new methods yield positive results. Fortunately we at Rutgers were in a position to do a statistical analysis of the effect that our new ``Workshop'' formats had on student performance, treating the standard (old format) sections as a control population.

The main study, by Profs. Hirsch and Weibel, concerns our mainstream first-semester Calculus course Math 135. This course has a standard 4-credit version, taught in a traditional format, and a 5-credit version, which features a weekly small-group Workshop.

Our results may be summarized as follows. After adjusting for prior skill, the students in the Workshop sections outperformed their peers in the standard sections by slightly over one letter grade (eg, from C to B or B+).

We did a separate study of Math 151-152. This is our Calculus sequence for Engineers and Science majors. The main point was to compare the 5-credit workshop sections to our 4-credit sections, in which the workshops and homework review were combined into one 80-minute class. In that study we found a difference of half a letter grade on the common final (after adjusting for prior skill level). This suggests that even combining homework review with a workshop improves final exam scores, and that the new formats are indeed an improvement upon the old standard teaching format.

Here are some photos of Calculus 151 Workshops (section 73) were taken in November, 2000: 1 2 3 4

The second half of that study followed the students into the second semester, Math 152. Here the best indicator was the score on the first-term final exam. The students in the workshop format both semesters scored a full letter grade higher than the control group (standard sections both terms). Students who switched from the standard section to the workshop format scored half a letter grade better (C+ to B) than the control group. Surprisingly, students who switched from workshops to standard sections scored a full letter grade lower than the control group. This drop was especially noticeable among the weakest students.

Finally, the second study analyzed the effectiveness of our EXCEL format, a 6-credit version of Calculus which has three full periods per week devoted to workshops. Although students were placed into in EXCEL based on low placement scores, they collectively scored as well in the first semester and half a grade higher in the second semester than the control group. For a student with these low placement scores, this meant a difference between an F and a C+ in the first semester.

Effectiveness of WeBWork, a web-based homework system

This is a separate statistical study by Lew Hirsch and Charles Weibel. In Spring 2001, Rutgers implemented a web-based homework system called WeBWork for its general first-semester Calculus course (Math 135), which has about 2,000 students each fall. The students take a common final exam at the end of the semester. Beacause of software limitations, only two thirds of all student were in sections requiring WeBWork problems during Fall 2001, while the rest were in traditional sections. This made it possible to treat the semester as a controlled experiment, in order to find out how effective this new system is.

Overall, we found a small but significant difference in performance on the final exam. The main problem we found was that many students in WeBWork sections just didn't do the problems, even though the web assignments were worth 5-10% of the course grade. If we eliminate students who were assigned WeBWork problems but attempted fewer than half the problems, the WeBWork sections did a half letter grade better than the control group; the average grade rose from C+ to B. The students in WeBWork sections who didn't attempt half the problems averaged a C.

There were thee different sub-populations in the class, with different profiles. Slightly over half were incoming first-year students; here there was a strong relation between how well students did on the WeBWork assignments and how they did on the final exam.

About 7% of the total were students who were repeating Calculus. There was no correlation at all between WeBWork assignments and final exam scores for this group. The remaining 25% of the clas consisted of upper-class students who had not previously taken Calculus at Rutgers. Their behavior was intermediate between the other groups.

Charles Weibel / weibel @ math.rutgers.edu / Summer, 2001