Observing assessment for learning in action: Piloting an observation tool to inform teacher assessment learning and research

Intro / Project description

In New  Zealand (NZ) we  lack valid  and  reliable  evidence-based tools with which to observe teachers’ formative assessment practices. To date, classroom assessment observation tools have mainly been derived theoretically and/or developed within particular curriculum areas. In this project we will pilot and modify an observation tool, the Developing and Evaluation Measures of Formative Assessment Practice (DEMFAP), developed in the USA from comprehensive video evidence, and evaluate its usefulness and suitability for the NZ primary school teaching context. Aims The aim of this project is to bring together teacher and university partners to implement, adapt and evaluate the DEMFAP observation tool for possible use in NZ primary classrooms.  Such a tool could be useful for improving and researching assessment for learning (afl) practice.

Aims

The aim of this project is to bring together teacher and university partners to implement, adapt and evaluate the DEMFAP observation tool for possible use in NZ primary classrooms.  Such a tool could be useful for improving and researching afl practice.

The following research questions will guide the project:

1. How well does the existing DEMFAP observation tool capture NZ teachers’ afl practices?
2. What  modifications  and  revisions  are  necessary  to  make  the  tool  useful  and relevant  in  the  NZ primary school context?
3. How effective is the modified tool at capturing critical aspects of NZ teachers’ practice for research purposes?
4. What are teachers’ perceptions about the efficacy and usefulness of the modified tool for improving their assessment capability?

Why is this research important?

Research evidence strongly supports embedding afl within classroom teaching in order to promote agentic learning for all students.  This notion is consistent with the aims of the NZ Curriculum.  Furthermore, as students  are  increasingly  expected  to manage  themselves  as  learners  and  work  collaboratively,  often  in flexible learning spaces, teachers need to build students’ afl skill in support of these approaches.  Although several projects have investigated how teachers’ and students’ afl practice can be improved, to date we do not have any observation tools capable of reliably capturing afl practice. Such tools are needed in order to assist teachers to gain information about how they implement formative strategies, and thus, how they might grow their expertise.  Additionally, a reliable and practical tool might also be used to assist preservice teachers to ‘see’ their developing afl practice and for researchers to measure assessment capability.

What we plan to do

During  the  first  three  months  of  2019  we  will  carefully  map  the  DEMFAP  tool  against  the  five  main  aflpractices known to underpin teacher assessment capability and as a team familiarise ourselves with the DEMFAP tool and its use.  Eight teachers, four from each school, will then use the DEMFAP to observe each others’ lessons  and  investigate  how  the  information  it  provides  can  be  used  to  enhance  their  practice.  Three cycles of use, data collection and analysis will take place during the year; cycle one will focus on tool familiarisation and use, cycle two on adaptation and use, and cycle three on revision, use and evaluation.

Data

In  each  cycle,  the  data  will  comprise  the  observation  records  for  each  lesson;  video recordings of lessons; reflective conversations with teachers about the use and usefulness of the tool; interviews with the teachers and the observers about the use and usefulness of the tool; and the data collected at regular teacher and researcher meetings to discuss the tool, it’s usefulness and possible modifications.

Analysis

To  answer  research  questions  1  and  2 in  each  cycle, data  will  be  analysed  for  inter-observer reliability, usability and how well the tool captures information afl practices. Inductive analysis of the interview and meetings data using the constant comparative method of open coding, axial  coding  and  deriving  themes  will  be  used  to  answer  research  questions  3  and  4.    This  will enable us to understand what teachers learnt about their afl practice, what needs to be modified to improve the efficacy of the tool and more about how the tool might be useful in the NZ context.

Our partners:

Principals and teachers from Edendale Primary School and Devonport Primary School in Auckland will work with the university team to pilot an observation tool designed to document teachers’ formative assessment practices used in assessment for learning (afl).These schools have a commitment to using afl and see development of an observation schedule as a tool they can use in professional learning and improvement.

Contact details:
Associate Professor Mary Hill
Learning, Development & Professional Practice
Education and Social Work
The University of Auckland
DDI: + 64 9 623 8899, EXT: 48630
mf.hill@auckland.ac.nz