Tag Archives: Assessment

How To Design A 21st Century Assessment

Contemporary curriculum design involves multiple facets: engaging 21st Century skills, using digital tools, collaborating with others around the globe, performance tasks, and more. Getting these design elements into a teacher’s current curriculum demands that teachers create professional habits around Replacement Thinking.

Four considerations for Replacement Thinking around assessments. In a nutshell, those considerations include:

  1. Students must demonstrate what they’ve learned. Whatever they create with digital tools should still represent what students were to learn. The assessment shouldn’t tell you more about their use of a tool than it does about the student’s work using the tool.
  2. Students should demonstrate content proficiency and sophistication. Their new product should reflect the content knowledge that they’ve learned and the multiple cognitive zones they participated in during the learning process.
  3. Students should be frequently reflecting on their choices. Students should be able to articulate and defend their tool choices, content inclusion, and degrees of audience interaction and how those choices affected the resulting product.
  4. Students must give credit where credit is due. They should know about copyright, Creative Commons licensing, and how to search for and use appropriate content, giving attribution for the media resources they use.

And while assessments are the focus of this blog post, replacement thinking can be applied across the curriculum: in instructional strategies, classroom activities, or in formative data collection using tools such as Kahoot, Socrative, or Google Forms. To help you start thinking about Replacement Thinking, I’d like to offer the following action steps to bring more contemporary ideas into your own professional practices:

  • Action Step 1: Stop thinking technology first.
  • Action Step 2: Give students authentic choice in how they will demonstrate their learning.
  • Action Step 3: Help students seek feedback from other students, other educators, and experts in the field.
  • Action Step 4: Provide always-on, asynchronous access to that which is being assessed.

Problem and Issues: What does 21st century learning and assessment look like?

How about Objective and Assessment: (Do technology-based options enhance or detract from student understanding of content? Data from student surveys will be evaluated to determine student engagement in project. Data from student surveys will determine student disposition at three stages of the process.)

21st-century-assessment-fi.jpg

  1. 21st Classroom Assessment
  2. 21st Century Skills_Assessment
  3. Guideline 21st Assessment
  4. Characteristic 21st Century Assessment
Advertisements

Using Biggs’ Model of Constructive Alignment in Curriculum Design

The main theoretical underpinning of the outcomes-based curriculum is provided by Biggs (2003). He calls the model constructive alignment which he defines as:

…coherence between assessment, gt strategeachinies and intended learning outcomes in an educational programme. (McMahon & Thakore 2006)

source: http://www.ucdoer.ie/index.php/Using_Biggs%27_Model_of_Constructive_Alignment_in_Curriculum_Design/Introduction

As currently articulated, the model is attributed to Biggs (2003, 1999) but the essentials were formulated by Tyler (1949) some 50 years earlier – and elaborated in the 1980s by Shuell (1986). At its most basic, the model requires alignment between the three key areas of the curriculum, namely, the intended learning outcomes, what the student does in order to learn, how the student is assessed. This is expressed in Figure 1 with a concrete example given as Figure 2.

Figure 1: A Basic Model of an Aligned Curriculum.

"Figure 1: A Basic Model of an Aligned Curriculum."

Figure 2: An Example of Constructive Alignment in a Curriculum

Examples of Alignment from Different Modules

Learning Outcomes:
On completion of this module students should be able:
Assessment Methods Teaching/Learning Activities
To identify the main signs and symptoms of multiple sclerosis. Multiple Choice Questions Lecture on various signs/symptoms,In class exercises/quizzes on terminology.
To formulate end products using selected ingredients Poster Display 15%Presentation of end product 85% Lecture presenting case studies of the design history of some market leaders.Students plan own project and present as poster.

Student projects on food formulation.

To develop and identify an area for research in the discipline 1,000 word research proposal Presentation of examples of research questions,Student discussion groups on research areas.
To demonstrate effective presentational skills In-class graded presentation Practices sessions in the class,Peer-assessment, using set criteria, of others in class.
Title of Module: Evaluating and Reflecting on your Teaching.
Outcomes
On completion of this module you should be able to:
Assessment
Critically reflective written report containing the following:
Teaching / Learning Activities
Monitor, evaluate and reflect on your teaching and the learning of your students Evidence of having completed the prescribed mentoring – observation cycleA reflective statement of personal and professional gains made from the peer observation process Introductory Group Tutorial – Revision of critical reflection theory (from previous modules).Seminar: Introduction to Peer Observation and the use of a Learning Contract.

Peer mentor sessions.

Use a range of methods to gather student feedback. Evidence of having received and responded to student feedbackA reflective statement of what has been achieved as a result of gathering feedback from students. Workshop:Methods of Gathering Student Feedback

Project: Collecting Student Feedback (using a variety of methods)

Contribute to the debate on the links between research and teaching. Formatively assessed by tutor comments in forum. (In preparation for formal assessment of this outcome in a future module.) On line forum

Biggs actually suggests that teaching and learning activities are designed second and the assessment regime third (page 30). If this sequence is adopted, it is important that activities are designed which enable students to learn how to demonstrate achievement at the highest level described by the outcomes. This can be done by focusing on the verbs within the outcomes that express “the very best understanding that could reasonably be expected” (page 28). (See Figure 3)

Appropriate verbs can be discovered or derived by relating the model to a learning taxonomy. The two most commonly used are that devised by Bloom (1956) as revised by Anderson et al (2001) and that devised by Biggs & Collis (1982). (See Figure 4)

Figure 3: Adapting the Model to Allow for Differential Levels of Achievement.

"Figure 3: Adapting the Model to Allow for Differential Levels of Achievement."

Figure 4: Relating the Constructive Alignment Model to Learning Taxonomies.

Bloom’s Taxonomy
(As revised by Anderson et al 2001)
Biggs’ Proposed Levels of Attainment Biggs & Collis’ SOLO Taxonomy
Synthesis / Creation
design, organise, formulate, propose.Evaluation
judge, appraise, evaluate, compare, assess.
A: The very best understanding Extended Abstract Thinking:
theorise
generalise
reflect
evaluate
Analysis
distinguish, analyse, calculate, test, inspect.Application
apply, use, demonstrate, illustrate, practice.
B: Highly Satisfactory Relational Thinking:
explain
analyse
compare
apply
Comprehension
explain, describe, discuss, recognise.
C: Quite Satisfactory Multi-structural Thinking
classify
comment upon
Knowledge
define, list, name, recall, record
D: Just a Pass Uni-structural
state
describe
E: Fail Pre-structural

A better fit with what Biggs says elsewhere in his book, however, is that the assessment regime needs to be thought out before the teaching and learning activities. This is because for students, assessment defines what is important in the curriculum and they will learn what they think will be assessed.

As Biggs put it:

… students learn what they think they will be tested on. This is backwash, when the assessment determines what and how students learn more than the curriculum does. In a poorly aligned system, where the test does not reflect the objectives, this will result in inappropriate surface learning. (Biggs 2003: 140)

Biggs notes that if the assessment regime does not properly reflect curriculum objectives then the result will be inappropriate “surface” learning. He then goes on to propose that educators use the inevitability backwash to secure effective educational reform.

You can’t beat backwash, so join it. Students will always second-guess the assessment task and then learn what they think will meet those requirements. But if those assessment requirements mirror the curriculum, there is no problem. Students will be learning what they are supposed to be learning. Ibid: 210.

This concept of backwash is a key element of, and justification for, the adoption of Bigg’s Model of Constructive Alignment because it is validated by a great deal of independent research (Atkins et al 1993, Ramsden 1992, Scouller 2000).

This does not, however, in anyway diminish the importance of the other two components of the curriculum. Any review or revision of any one of the three components of an aligned curriculum requires a matching review or revision of the other two. Where a curriculum is not aligned – i.e. where there is a discontinuity between any two of the components – it is likely that there will be a mismatch between intention and product.

At a more complex level, constructive alignment requires a balance and synergy between:

  • the professional goals of the teachers
  • the wants and needs of the students
  • the curriculum
  • the teaching methods used
  • the assessment procedures used and the method or report results
  • the psychological and social climate of the classroom (learning milieu)
  • the psychological and social climate of the institution.

Each of these components needs to work towards the common goals.

Imbalance in the system will lead to poor teaching and surface learning. Non alignment is signified by inconsistencies, unmet expectations, and practices that contradict what we preach (Biggs 2003: 26)

Figure 5: Examples of Alignment from Different Modules

Learning Outcomes:
On completion of this module students should be able:
Assessment Methods Teaching/Learning Activities
To identify the main signs and symptoms of multiple sclerosis. Multiple Choice Questions Lecture on various signs/symptoms,In class exercises/quizzes on terminology.
To formulate end products using selected ingredients Poster Display 15%Presentation of end product 85% Lecture presenting case studies of the design history of some market leaders.Students plan own project and present as poster.

Student projects on food formulation.

To develop and identify an area for research in the discipline 1,000 word research proposal Presentation of examples of research questions,Student discussion groups on research areas.
To demonstrate effective presentational skills In-class graded presentation Practices sessions in the class,Peer-assessment, using set criteria, of others in class.

SOLO Taxonomy (Biggs & Collis, 1982)

The Structured Overview of Learning Outcomes, SOLO Taxonomy (Biggs & Collis, 1982), provides a common understanding of the learning process through an overview of cognitive learning outcomes. We have introduced this taxonomy to schools with students from 5 to 18 years of age.

SOLO provides criteria that identify the increasing complexity of student performance for understanding when mastering new learning (Biggs 1999, p.37). It is content independent and thus is useful as a generic measure of understanding across different disciplines. In our experience teachers using SOLO can easily, and reliably identify ascending cognitive complexity in individual and collective student learning outcomes.

SOLO describes five levels of student understanding, Refer Figure below.

solo_taxonomy
SOLO 5 Levels of Understanding

At the //prestructural level// of understanding, the student response shows they have missed the point of the new learning. At the //unistructural level,// the learning outcome shows understanding of one aspect of the task, but this understanding is limited. For example, the student can label, name, define, identify, or follow a simple procedure. At the //multistructural level,// several aspects of the task are understood but their relationship to each other, and the whole is missed. For example, the student can list, define, describe, combine, match, or do algorithms. At the //relational level,// the ideas are linked, and provide a coherent understanding of the whole. Student learning outcomes show evidence of comparison, causal thinking, classification, sequencing, analysis, part whole thinking, analogy, application and the formulation of questions. At the //extended abstract level,// understanding at the relational level is re-thought at a higher level of abstraction, it is transferred to another context). Student learning outcomes at the extended abstract level show prediction, generalisation, evaluation, theorizing, hypothesising, creation, and or reflection.

Using visual symbols to represent levels of understanding in SOLO means that coding for complexity of thinking can be undertaken by both student and teacher, allowing “where should we go next?” decisions and thinking interventions to more accurately target student learning needs.

Topic 0020: Assessment vs Evaluation

What is the difference between “assessment” and “evaluation?”

Assessment is the process of objectively understanding the state or condition of a thing, by observation and measurement. Assessment of teaching means taking a measure of its effectiveness. “Formative” assessment is measurement for the purpose of improving it. “Summative” assessment is what we normally call “evaluation.”

Evaluation is the process of observing and measuring a thing for the purpose of judging it and of determining its “value,” either by comparison to similar things, or to a standard. Evaluation of teaching means passing judgment on it as part of an administrative process.

Ideally, a fair and comprehensive plan to evaluate teaching would incorporate many data points drawn from a broad array of teaching dimensions. Such a plan would include not only student surveys, but also self-assessments, documentation of instructional planning and design, evidence of scholarly activity to improve teaching, and most importantly, evidence of student learning outcomes.

Reference: http://www.itlal.org/?q=node/93