Jan Mokros, Mary Berle-Carman, Andee Rubin, and Tracey Wright
TERC, Cambridge, MA
December, 1994
Summary
During the 1992-93 school year, we conducted an evaluation of the impact of the NSF-funded Investigations curriculum on third- and fourth-graders' mathematical understanding. A pre-post test, comparison group design was used, with 96 students completing the pre-test in the fall and the post-test in the spring. The assessment included a written test stressing calculation skills, as well as hour-long individual interviews which examined students' understanding of concepts involving number, data, and geometry. The interview results show that students in the Investigations group made substantially greater gains than students in the comparison group, particularly for items involving understanding of number and geometry. On the written test, both comparison and Investigations students made substantial gains. We conclude that the Investigations curriculum has the potential to substantially deepen students' mathematical understanding. At the same time, it does not seem to stand in the way of performance on traditional tests of computation skills.
The Investigations Curriculum
Investigations in Number, Data, and Space is a new K-5 curriculum presented through a series of teacher books rather than traditional textbooks. Each book explores major mathematical ideas while communicating mathematics content and pedagogy to teachers. Depth in mathematical thinking rather than a superficial exposure to a series of fragmented topics is emphasized. Students find more than one solution to many problems they work on, and they invent their own strategies and approaches rather than relying on memorized procedures. They choose from a variety of concrete materials and appropriate technology, including calculators, as a part of their everyday mathematical work. While working as a whole class, individually, in pairs, and in small groups, students express their mathematical thinking through talking, drawing, and writing.
Method
Participants During the 1992-93 school year, we examined third and fourth graders' learning as they were participating in the Investigations curriculum. We also examined the mathematical learning of a comparison group of students who were participating in standard mathematics classes. Children from seven Investigations classrooms, along with students from five comparison classes were involved in the study. Comparison classes were chosen to have equivalent socioeconomic and student population characteristics as the Investigations classrooms. The classrooms were located in five Massachusetts communities, including schools in urban, suburban, and rural communities. Students were diverse with respect to race, gender, background characteristics, first language, and mathematical skills. Eight randomly selected students from each classroom were involved in the study.
Assessment Instruments Two instruments were developed--a written test and a set of interview items. The written test included items similar to those found on many standardized tests at these grade levels. These included addition, subtraction, multiplication and division computation problems presented in numerical format, word problems, filling in missing numbers on a hundreds chart, placing familiar fractions on a number line from 0 to 1, and figuring out the total cost of three items on a menu. A TERC interviewer administered the untimed test to small groups of students in each school. Most students completed it in less than 25 minutes.
The interview tasks were constructed by a team of assessment specialists and curriculum developers at TERC to reflect the goals stressed by the NCTM Standards and the Investigations curriculum. The tasks included content in the areas of number, geometry, and data. As part of solving these problems, students were asked to estimate, predict, use manipulatives and calculators, explain their strategies, and show how they were thinking through the use of drawings and constructions. The interviews consisted of several tasks, each taking about 5 to 8 minutes to complete. Interviews were conducted individually by a member of the assessment team.
Analysis A three-point rubric was developed to score each interview item. The factors considered in analyzing students' responses included:
- the choice of an appropriate strategy to solve the problem
- completeness and accuracy of the solution
- clarity of explanation and/or demonstration of how to solve the problem
- flexibility in dealing with new constraints introduced into the task.
The written test, unlike the interview tasks, were scored only as correct or incorrect. As on many traditional tests, no points were awarded on the written test for good strategies - only for arriving at correct answers. During scoring of all items, assessment team members did not know whether students' work came from the Investigations or comparison group. We achieved a high degree of inter-rater reliability (95% agreement) on the scoring of interview tasks.
Results
Written Test Not surprisingly, students in both third and fourth grades performed significantly better on the post-test than on the pre-test (p < .0001). The amount of change was similar for the Investigations and the comparison group. The differences between the groups were not statistically significant; students in the two groups made comparable gains.
Interview Tasks The Investigations students made significantly greater gains than the comparison group in solving the interview problems. When third and fourth graders' scores on interview tasks are combined, we again find that both groups gained significantly over the course of the year (p < .0001). The pattern of gains is described in the table.
| Gains for Investigations group only | Gains for comparison group only |
|---|---|
|
Grade 3. 2 items - geometry and multiplication |
Grade 3. none |
| Gains for Both Groups | No Gains |
|
Grade 3. 5 items - data (2 items), fractions, number sense, geometry |
Grade 3. 1 item - graphing |
Discussion
Overall, the Investigations group students made fairly substantial gains on all but three interview tasks as well as on the written test. Two of these three items had a ceiling effect, that is, students in both groups scored so well on the pre-test that there was little room for improvement on the post-test. There were no tasks in which only the comparison group showed gains, while there were six interview tasks in which only the Investigations group showed gains. As anticipated, students using this new curriculum were at no disadvantage in solving "traditional" arithmetic problems. Moreover, students using Investigations were at a distinct advantage when it came to solving interview tasks that involved more conceptual work.
Most interesting was the fact that students in the Investigations group performed particularly well on the number tasks in the interview. Many of these tasks involved familiar arithmetic, but students needed to choose and carry out strategies based on a good understanding of number relationships and operations. For example, in the Elephant Problem (fourth grade) students figure out how many 60 pound dogs would weigh the same as a 3000 pound elephant. We found that some students knew that they should choose an operation and perform it, but didn't know which operation to use. On the other hand, students who were developing good number strategies approached the task by building up chunks of 60's in order to find out how many 60's would equal 3000. We were hoping to see this kind of strategy, and we found it more often among students in the Investigations group than in the comparison group.
We also found stronger gains in the Investigations group on the 3-D geometry tasks (make a pattern for a box that will hold 3 cubes; figure out how many blocks are in a drawing of a rectangular solid). These tasks emphasized moving between 2-dimensional representations and 3- dimensional space. Both groups made comparable gains on the data tasks.
Conclusion
Students in Investigations classrooms performed comparably to those in comparison classrooms in doing traditional computation tasks. They made significant progress in areas of mathematics emphasized by the Investigations curriculum, in particular in developing and applying number sense, understanding the structure of the number system, and representing and visualizing 3- dimensional space. This study was carried out in Investigations classrooms where both students and teachers were using draft versions of the curriculum for the first time, while the comparison classrooms were using approaches that were familiar to them. We believe that as teachers make Investigations their own, and become more comfortable with how mathematics teaching and learning are changing, the impact of the curriculum on student understanding will be even greater.
Acknowledgements
The research described in this paper was supported, in part, by the National Science Foundation Grant No. ESI-9050210. Opinions expressed are those of the authors and not necessarily those of the Foundation.