Tag Archives: evaluation

Assessment, Measurement & Evaluation

Measurement refers to the process by which the attributes or dimensions of some physical object are determined. One exception seems to be in the use of the word measure in determining the IQ of a person. The phrase, “this test measures IQ” is commonly used. Measuring such things as attitudes or preferences also applies. However, when we measure, we generally use some standard instrument to determine how large, tall, heavy, voluminous, hot, cold, fast, or straight something actually is. Standard instruments refer to physical devices such as rulers, scales, thermometers, pressure gauges, etc. We measure to obtain information about what is. Such information may or may not be useful, depending on the accuracy of the instruments we use, and our skill at using them. There are few such instruments in the social sciences that approach the validity and reliability of say a 12″ ruler. We measure how big a classroom is in terms of square feet or cubic feet, we measure the temperature of the room by using a thermometer, and we use an Ohm meter to determine the voltage, amperage, and resistance in a circuit. In all of these examples, we are not assessing anything; we are simply collecting information relative to some established rule or standard. Assessment is therefore quite different from measurement, and has uses that suggest very different purposes. When used in a learning objective, the definition provided on the ADPRIMA site for the behavioral verb measure is: To apply a standard scale or measuring device to an object, series of objects, events, or conditions, according to practices accepted by those who are skilled in the use of the device or scale. An important point in the definition is that the person be skilled in the use of the device or scale. For example, a person who has in his or her possession a working Ohm meter, but does not know how to use it properly, could apply it to an electrical circuit but the obtained results would mean little or nothing in terms of useful information.

Click here for a brief explanation of the different types of measurement scales. The information will give you a little more context for the preceding section.

Assessment is a process by which information is obtained relative to some known objective or goal. Assessment is a broad term that includes testing. A test is a special form of assessment. Tests are assessments made under contrived circumstances especially so that they may be administered. In other words, all tests are assessments, but not all assessments are tests. We test at the end of a lesson or unit. We assess progress at the end of a school year through testing, and we assess verbal and quantitative skills through such instruments as the SAT and GRE. Whether implicit or explicit, assessment is most usefully connected to some goal or objective for which the assessment is designed. A test or assessment yields information relative to an objective or goal. In that sense, we test or assess to determine whether or not an objective or goal has been obtained. Assessment of skill attainment is rather straightforward. Either the skill exists at some acceptable level or it doesn’t. Skills are readily demonstrable. Assessment of understanding is much more difficult and complex. Skills can be practiced; understandings cannot. We can assess a person’s knowledge in a variety of ways, but there is always a leap, an inference that we make about what a person does in relation to what it signifies about what he knows. In the section on this site on behavioral verbs, to assess means To stipulate the conditions by which the behavior specified in an objective may be ascertained. Such stipulations are usually in the form of written descriptions.

Evaluation is perhaps the most complex and least understood of the terms. Inherent in the idea of evaluation is “value.” When we evaluate, what we are doing is engaging in some process that is designed to provide information that will help us make a judgment about a given situation. Generally, any evaluation process requires information about the situation in question. A situation is an umbrella term that takes into account such ideas as objectives, goals, standards, procedures, and so on. When we evaluate, we are saying that the process will yield information regarding the worthiness, appropriateness, goodness, validity, legality, etc., of something for which a reliable measurement or assessment has been made. For example, I often ask my students if they wanted to determine the temperature of the classroom they would need to get a thermometer and take several readings at different spots, and perhaps average the readings. That is simple measuring. The average temperature tells us nothing about whether or not it is appropriate for learning. In order to do that, students would have to be polled in some reliable and valid way. That polling process is what evaluation is all about. A classroom average temperature of 75 degrees is simply information. It is the context of the temperature for a particular purpose that provides the criteria for evaluation. A temperature of 75 degrees may not be very good for some students, while for others, it is ideal for learning. We evaluate every day. Teachers, in particular, are constantly evaluating students, and such evaluations are usually done in the context of comparisons between what was intended (learning, progress, behavior) and what was obtained. When used in a learning objective, the definition provided on the ADPRIMA site for the behavioral verb evaluate is: To classify objects, situations, people, conditions, etc., according to defined criteria of quality. Indication of quality must be given in the defined criteria of each class category. Evaluation differs from general classification only in this respect.

To sum up, we measure distance, we assess learning, and we evaluate results in terms of some set of criteria. These three terms are certainly share some common attributes, but it is useful to think of them as separate but connected ideas and processes.

Here is a great link that offer different ideas about these three terms, with well-written explanations. Unfortunately, most information on the Internet concerning this topic amounts to little more than advertisements for services.

 

Advertisements

SOLO Taxonomy (Biggs & Collis, 1982)

The Structured Overview of Learning Outcomes, SOLO Taxonomy (Biggs & Collis, 1982), provides a common understanding of the learning process through an overview of cognitive learning outcomes. We have introduced this taxonomy to schools with students from 5 to 18 years of age.

SOLO provides criteria that identify the increasing complexity of student performance for understanding when mastering new learning (Biggs 1999, p.37). It is content independent and thus is useful as a generic measure of understanding across different disciplines. In our experience teachers using SOLO can easily, and reliably identify ascending cognitive complexity in individual and collective student learning outcomes.

SOLO describes five levels of student understanding, Refer Figure below.

solo_taxonomy
SOLO 5 Levels of Understanding

At the //prestructural level// of understanding, the student response shows they have missed the point of the new learning. At the //unistructural level,// the learning outcome shows understanding of one aspect of the task, but this understanding is limited. For example, the student can label, name, define, identify, or follow a simple procedure. At the //multistructural level,// several aspects of the task are understood but their relationship to each other, and the whole is missed. For example, the student can list, define, describe, combine, match, or do algorithms. At the //relational level,// the ideas are linked, and provide a coherent understanding of the whole. Student learning outcomes show evidence of comparison, causal thinking, classification, sequencing, analysis, part whole thinking, analogy, application and the formulation of questions. At the //extended abstract level,// understanding at the relational level is re-thought at a higher level of abstraction, it is transferred to another context). Student learning outcomes at the extended abstract level show prediction, generalisation, evaluation, theorizing, hypothesising, creation, and or reflection.

Using visual symbols to represent levels of understanding in SOLO means that coding for complexity of thinking can be undertaken by both student and teacher, allowing “where should we go next?” decisions and thinking interventions to more accurately target student learning needs.

Topic 0020: Assessment vs Evaluation

What is the difference between “assessment” and “evaluation?”

Assessment is the process of objectively understanding the state or condition of a thing, by observation and measurement. Assessment of teaching means taking a measure of its effectiveness. “Formative” assessment is measurement for the purpose of improving it. “Summative” assessment is what we normally call “evaluation.”

Evaluation is the process of observing and measuring a thing for the purpose of judging it and of determining its “value,” either by comparison to similar things, or to a standard. Evaluation of teaching means passing judgment on it as part of an administrative process.

Ideally, a fair and comprehensive plan to evaluate teaching would incorporate many data points drawn from a broad array of teaching dimensions. Such a plan would include not only student surveys, but also self-assessments, documentation of instructional planning and design, evidence of scholarly activity to improve teaching, and most importantly, evidence of student learning outcomes.

Reference: http://www.itlal.org/?q=node/93