Computers and Mathematical Assessment

Douglas H. Clements
State University of New York at Buffalo

My concept of myself as a student has suffered somewhat since I began (secretly) tanking my sixth-grader's science tests. I am fine on the mathematics, mind you, but the biological terms decimate my scores. How would you do, today, on a range of public school assessments? What does that mean?

Nature of Assessment in Mathematics

In far too many cases, we are testing, and therefore teaching, the wrong things. The long history of laments regarding knowledge will continue unabated unless we transform mathematics assessment. Reductionist pieces of information are in part the result of national standardized, instruments that test only that at the intersection of myriad rich and shallow curricula (Lipson, Faletti, & Martinez, 1990). Mathematics innovation cannot grow in such environments.

Instead, we need additional forms of assessment, including those called "authentic," "performance," or "direct." Such forms are based on tasks that range from simple but meaningful to complex, integrated and challenging. Most of these tasks should, to varying degrees, require reflection, emphasize important concepts, and demand persistence. School systems, especially those that score low, will find a way to "reduce the level" (van Hiele, 1986) of instruction to improve scores on any other type of assessment of "better" or "higher-order" mathematics (Baker, 1990; Frederiksen & Collins, 1989).

The purpose of this paper is to suggest ways that computers can help promote such assessment and therefore contribute to reform innovation in mathematics education.

Computer-enhanced Assessment

Computer may enhance assessment in a variety of ways. For example, typical feedback from printed tests is the delayed presentation of a score and rank. Computers can provide more timely and elaborated feedback. For example, if a problem involved calculating paint to cover a wall, the computer might show the difference between the given area and the area covered by the amount of paint the student calculated (Lipson et al., 1990). In such ways, computer-based assessment can be more instructionally powerful. There are, in addition, many ways computers can significantly alter assessment.

Adaptive Testing

A relatively simple, but still underutilized, form of computer-enhanced testing is adaptive testing (Lesh, 190; McKinley & Reckase, 1980). During testing, the computer selects items that are deemed appropriate for the student based on her or his previous responses. If the student's performance indicates that she or he will not be able to handle certain types of times, these are not presented. Computer testing also allows the efficient use of open-ended items and new questioning sequences rather than the restricted multiple-choice format. Multiple versions allow only summarizing outcomes. Errors of measurement are minimized, testing time shortened (from 1/2 to 1/3 of the time for conventional assessment), frustration reduced, reliability improved, and total test information gathered increased, including profiles of strengths and weaknesses rather than unidimensional scores. This type of assessment, then, takes less time away from, and contributes more to classroom instruction.

Generally, research has supported the efficacy of computer diagnosis and remediation of arithmetical errors. Future software developments should promote the transition from multiple-choice formats with a single correct choice to constructed-response questions and open-ended questions and problems that test higher level knowledge and skill. Eventually, such software might present many different kinds of problems (with different contexts generated to match the interests of each student), and track and interpret not only answers, but intermediate steps in solution attempts (Lipson et al., 1990).

In a similar vein, adaptive instructional systems perform an initial diagnostic assessment of each student (or skip this time-consuming process if prior information indicates minimal existing knowledge). Then assessment is continually updated and new prescriptions are made as a student works. The amount of instruction given is based on the student's progress towards the mastery of the objectives, which reduces off-task time (Tennyson, Christensen, & Park, 1984).

System such as this can help establish connections among related ideas, both within mathematics and across other subject matter areas. Systems that generate an array of interdisciplinary problems and track multiple concepts and processes may facilitate the development comprehensive interdisciplinary instruction (Lipson et al., 1990).

Increasing Intelligent Assessment

As systems become more intelligent, valid micro-assessments become achievable (Snow, 1989). Enhanced with a model of the learner (Frederiksen & Collins, 1989), computers can make more finely tailored assessment and instructional decisions. For example, Lesgold's (1988) tutor for troubleshooting electrical circuits continually updates its student model in relation to a previously specified goal structure and designs the next task to be presented based on a list of constraints representing other relevant initial states of learners, such as initial mathematical ability. For example, the following decision might be made: A student is ready to move the next specified level of complexity in resistor networks, but is weak in arithmetic skill, so the voltage computations will be kept mathematically simple. In this way, the tutor circumvents some aptitude weaknesses while working on other transitions.

For mathematics, systems should begin to be developed in domains that research has explored in detail (Lipson et al., 1990). For example, Marshall (1990) has applied schema theories to build computer-based assessments of solving arithmetic story problems. A variety of item types provided distinct information that judged the level of understanding demonstrated by different responses.

Such systems eventually may be able to provide adaptive assessment and instruction, and also analyze and interpret a student's performance beyond correctness; make sophisticated analyses of multiple step problems; incorporate techniques of dynamic assessment; allow assessment of higher-order thinking, circumventing the need for beginners to automatize skills; maintain a dynamic model and representation of the student's knowledge and skill, access a data base of mathematics relevant to the curriculum, including applications outside of the school context; provide progress maps depicting the student's learning over time, suggesting possible areas for additional study, and mapping out the "landscape" ahead of the knowledge to be gained, including possible points of difficulty; and make instructional suggestions (Lajoie, 1990; Lipson et al., 1990).

Simultaneous Group and Individual Assessment

Though research is limited, there is also significant potential in new systems that allow multiple students to communicate with a teacher. For example, each member of a class might enter their response to a problem, which would all be instantly transmitted to the teacher. The teacher can examine, display, and discuss various solutions with the class, monitor the responses of the class as a whole, of groups, or of individuals, and store information at each of these aggregate levels. Students might use keypads or calculators to enter their solutions.

"Doing Mathematics"

To assess student's ability to fully "do mathematics," the state of the art still requires subjective assessments that include performance on extended tastes (Frederiksen & Collins, 1989). Computer traces of students work processes can help document such performance. However, the computer must be integrated into a larger assessment effort. Further, the computer can make additional contributions, many intimated previously. For example, they can be used to create simulations and micro worlds that are rich source of problems. Work with Logo and other environments has provided "windows to the mind" for many teachers and researchers (Chazan, 1991; Clements & Meredith, 1993; Weir, 1987). Weir, for example, showed that children's first choices of number and input to the Logo "forward" command revealed much about their number sense. Teachers similarly learn much about their students exploring mathematics concepts, including fraction, with Turtle Math (Clements & Meredith, 1994). In one case, students had completed a week of instruction on fractions, but when encountering fractions on the computer, they relied on instinct (Sarama, 1995). For example, Linda was playing Berry Good Meal, an activity in which she was to enter a forward command that enabled the turtle to eat berries placed on a line segment somewhere from 0 to 1 (in Turtle Math, the turtle's step size can be changed: in this activity, it was 200 pixels. Linda guessed 2/3; when the turtle did not go far enough, she wanted to try 2/4. When that did not work, she said, "Of course," and mentioned that 2/4 is the same as 1/2. Though she originally was trying to change the denominator by rote to make the fraction smaller, the feedback from the turtle encouraged her to reflect on what she had entered and make quantitative sense of the fractions. This teacher also observed that computer environments encouraged students to try their own algorithms and thus become mathematical empiricists, rather than merely receivers of knowledge from authorities. When trying to play Berry Good Meal in a way in which the input to the fd command was a sum, two girls, Carrie and Heather, tried 1/3 + 1/3 for what they thought was 1/3, with Heather reasoning 1/3 + 1/3 is 2/6 (adding numerator an denominator) which reduces to 1/3. Carrie disagreed, but Heather was insistent that her logic was correct, so they tried it. Carrie used the feedback from the turtle as proof that Heather was wrong and Heather worked to reconcile her original idea with the feedback (Sarama, 1995). Such insights into children's epistemological actions is essential for the teacher attempting to implement mathematics education reform.

Recommendations

1. Change the basic approach to assessment.

De-emphasize reductionist, indirect testing in favor of direct assessment so that systems cannot reduce the level of educational goals.
Eliminate paper-and-pencil tests from the earliest years.
Eliminate unidimensional, passive indicators in favor of assessment as an integral part of complex, dynamic, self-regulating organic system.
Eliminate most separate assessments; most assessment should occur within instructional activities.

2. Recognize that innovation in mathematics education can be substantially enhanced through technology. Use computer for

Enhancing feedback
Adaptive testing
Increasing intelligent assessment
Simultaneous group and individual assessment
Providing environments for "doing mathematics"

3. Consider the need to improve teacher education on assessment, the use of computers, and their synthesis.

References

Baker, E. L. (1990). Developing comprehensive assessments of higher order thinking. In G. Kulm (Ed.) Assessing higher order thinking in mathematics (pp. 7-20). Washington, DC: American Association for the Advancement of Science.

Chazan, D. (1991). Research and classroom assessment of students' versifying, conjecturing, and generalizing in geometry. Unpublished manuscript, Michigan State University, East Lansing, MI.

Clements, D. H., & Meredith, J.S. (1993). Research on Logo: Effects and efficacy. Journal of Computing in Childhood Education, 4, 263-290.

Clements, D. H., & Meredith, J.S. (1994) Turtle math [Computer program]. Montreal, Quebec: Logo Computer Systems, Inc. (LCSI).

Frederiksen, J.R., & Collins, A. (1989). A systems approach to educational testing. Educational Researcher, 18(9), 27-32.

Lajoie, S.P. (1990, April). Computer environments as cognitive tools for enhancing mental models. Paper presented at the meeting of the American Educational Research Association, Boston, MA.

Lesgold, A. (1988). Toward a theory of curriculum for use in designing intelligent instructional systems. In H. Madl & A. Lesgold (Eds.), Learning issues for intelligent instructional systems (pp. 114-137). New York: Springer.

Lesh, R. (1990). Computer-based assessment of higher order understandings and processes in elementary mathematics. In G. Kulm (Ed.), Assessing higher order thinking in mathematics (pp. 81-110). Washington, DC: American Association for the Advancement of Science.

Lipson, J. I., Faletti, J., & Martinez, M.E. (1990) Advances in computer-based mathematics assessment. In G. Kulm (Ed.), Assessing higher order thinking in mathematics (pp. 121-134). Washington, D.C.: American Association for the Advancement of Science.

Marshall, S. P. (1990). The assessment of schema knowledge for arithmetic story problems: A cognitive science perspective. In G. Kulm (Ed.), Assessing higher order thinking in mathematics (pp. 155-168). Washington, DC: American Association for the Advancement of Science.

McKinley, R. L., & Reckase, M.D. (1980). Computer applications to ability testing. AEDS Journal, 13, 193-203.

Sarama, J. (1995). Redesigning Logo: The turtle metaphor in mathematics education. Unpublished doctoral dissertation, State University of New York at Buffalo, Buffalo, NY.

Snow, R. E. (1989). Toward assessment of cognitive and conative structures in learning. Educational Researcher, 18(9), 8-14.

Tennyson, R. D., Christensen, D.L., & Park, S. I. (1984). The Minnesota adaptive instructional system: An intelligent CBI system. Journal of Computer-Based Instruction, 11, 2-13.

Van Hiele, P.M. (1986). Structure and insight. Orlando, Fl: Academic Press.

Weir, S. (1987) Cultivating minds: A Logo casebook. New York, NY: Harper & Row.

Other Articles by Douglas H. Clements:

7 Ways to Add Math to Everyday Play

Analyzing Children's Length Strategies with Two-Dimensional Tasks: What Counts For Length?

Building Blocks of Early Childhood Mathematics PDF

Building Blocks for early childhood mathematics PDF

Building Blocks for Young Children's Mathematical Development PDF

A Case for a Logo-Based Elementary School Geometry Curriculum

Challenges for Teachers Attempting to Integrate a Mathematics Innovation

Computers in Early Childhood Mathematics PDF

Computers Support Algebraic Thinking