The
courses we have today have grown out of an international academic
community that has guided the development of course objectives,
methods of delivery and assessment techniques. They are products of
a vast community of builders, distributed across the globe and
through time. It is something of a surprise that this community is
not more frequently accessed in evaluation strategies.
Competition
between different institutions and the labour-intensive nature of
traditional onsite peer evaluation have been barriers to harnessing
the rich expertise of the academic community. Today, the
collaborative benchmarking culture and innovative collaborative tools
mean that we are more flexible in our approach and comfortable in
seeking guidance from peers.
Like
any form of feedback, peer review can happen at various scales.
Traditionally, experts have been hired to perform formal, top down,
whole of course reviews that are disruptive, very time intensive and
depend on the expertise and reputation of the academic reviewer.
While useful, the ’inquiry style' review is costly, time intensive
and can become controversial and adversarial - it may not be as
effective as a range of different peer review techniques including:
- Feedback on a unit outline (which may also be part of a pre-moderation process), providing input on specific issues such as assessment and articulation of outcomes. This can be resourced effectively on a quid pro quo basis.
- Benchmarking communities setting common expectations for learning outcomes within a discipline. Collaborative design does not preclude different institutions from competing on the basis of their individual strengths
- Building communities of practice around teaching within a discipline or focussed on a specific issue such as work integrated learning. Online communities can provide rapid feedback and an effective way to crowdsource ideas.
- Peer observation and feedback on specific issues such teaching techniques or design of class activities. These reports can be short and relatively informal but will be useful for feedback and for individual teaching portfolios.
We
are generating new ideas to connect units into course frameworks and
to build emergent structures out of shared outcomes and approaches.
The individual unit of study had become somewhat atomised in the
contemporary university, conceptual walls that were designed to
protect academic autonomy sometimes lead to isolation in the larger
scales and changing staff conditions that universities operate on
today. Peer feedback from within a organisation or even from without
can be a way of re-connecting units and cultivating academic
communities that support the academic responsible for a that unit.
As
indicated previously, there can be some overlap between this sort of
evaluation and moderation
processes. These can be usefully combined to ensure quality of
assessment through moderation but to contribute to unit or course
development through peer feedback. Pre-moderation of course guides
is a useful way of ensuring quality of assessment before a semester
has begun as it can detect problems before students are set
assessment tasks and this process also enhances overall learning
design.
Activating
Mentors
Academic
faculties have grown dramatically in size with the massification of
education and influx on new students, domestically and
internationally. This has led to a shortfall in effective mentoring
which has impacted on the academic communities built around course
offerings. Practice is enhanced by the ability to draw on the
experience and advice of mentors, yet many academics have never had
the opportunity to do this. Building a mentoring structure into an
evaluation system is a way of supporting both course development and
the staff who are responsible for it.
Course
mentors may be formally or informally placed, independent of school
management structures, to provide guidance for new teachers but also
a collegial and reflective critical friend for more experienced
staff. This has been the traditional role of professors in many
places but like other practice, much of this has been eroded by the
accelerated pace of university life and the growth of department
size. Evaluation can be a useful umbrella to bring back some of
these foundations of academic community.
Mentors
may also come from other sources such as industry, the professions or
the community. While these mentors may not have expertise in
educational design they may provide a useful grounding and be able to
provide advice about current practice. Some departments retain
adjunct professors to fill this kind of need.
Methods
- Outline review
- Content review
- individual feedback
- Portfolio of course artefacts
- Course mentor structure.
Key
Points
- The academic community is a rich source of effective feedback and we have a variety of technologies to make this process easier and less resource intensive
- Formal inquiry style peer review might be usefully replaced with a range of informal and less costly opportunities especially involving communities of practice and crowdsourcing
- Benchmarking provides opportunities for cross-institutional collaboration without precluding universities from competing on their individual strengths
- Feedback can be tied to the moderation process
- Mentors can support the foundations of a course community and support staff development
|
Outline
Review
In
a nutshell:
Peers
provide feedback on course/unit materials with a focus on
documentation such as outlines and assessment design. An outline
review can also be part of a pre-moderation process where the
reviewer checks the alignment of outcomes and assessment criteria.
As a feedback tool, this can be done under a paid contract or a
quid pro quo relationship.
Example
Questions:
Are
the assessment tasks consistent with overall outcomes?
How
does this course/unit benchmark with similar offering elsewhere?
Reporting:
Generally
a confidential report which is used internally.
Pros:
Takes
advantage of a different point of view, particularly where peer
reviewer is an experienced teacher.
The
process can be desk based, few extra costs or demands on peer
reviewer.
Cons:
It
is time consuming and a perfunctory review is probably worse than
no review at all.
Caution:
Peers
need to be chosen carefully and given instructions about the
parameters of the review process. They need to give feedback on
the choices actually made, not whether or not they would make the
same choices themselves. Giving good peer feedback is a skill
which should be cultivated in your own course group so that you
can engage more effectively with others.
Content
Review
In
a nutshell:
This
form of peer review focuses on the content aspect of curriculum.
It may be part of an outline review or it may take a more dynamic
form such as sharing of learning resources and materials, creation
of shared wikis, use of social bookmarking (such as learni.st) and
other ways of collaborative benchmarking.
A
textbook can be a form of collective knowledge review, although
this gets complicated by issues of individual credibility and
ownership.
Example
Questions:
How
is the knowledge of the field kept up to date?
Are
critical debates and opinions reflected in course materials?
Reporting:
The
best model of collaboration works publicly, transparently and is
shared openly.
Pros:
Allows
for broader benchmarking strategies
Crowd
sourcing of labour and reducing the redundancy of each program
working in isolation.
Cons:
Can
be a very indirect way of implementing change in your local
curriculum.
Caution:
In
the past universities tried to compete on having the best learning
resources and content, but in the internet world this becomes a
redundant way of thinking. It is better to collaborate on
content, preferably open education resources and then compete on
the distinctive local features of each institution.
Individual
Feedback
In
a nutshell:
Most
of the strategies in this manual are focussed at the course/unit
rather than the single teacher, but there remains a need for
feedback on an individual professional level. Peer observation
and feedback on teaching can be vital for individual professional
development and career progression and may also have some input
into overall course development where the individual is willing to
share.
Example
Questions:
Is
my verbal delivery clear and easily understood?
Does
the way that I structure and deliver class activities enhance
learning?
Reporting:
Generally
this sort of feedback is confidential, save where the individual
choses to share it with others, such as in a teaching portfolio or
promotion application. Reflection on this feedback is crucial in
demonstrating what has been applied.
Pros:
A
collegial way to share experience and to help peers, whether they
be junior, senior or at the same level of experience.
Can
detect problems that the individual is not conscious of.
Cons:
It
can be easy to be distracted by polished delivery and surface
issues rather than maintaining a focus on student learning.
Caution:
This
method generally requires the peer to be present at delivery and
to have a working knowledge of the unit objectives and approach.
Videotaping or including a peer reviewer by remote access may not
create an equivalent environment, unless the course is generally
delivered in that mode.
Students
should be made aware of the peer reviewer and their role as they
may be curious about the presence of a stranger, depending on
class size. |
No comments:
Post a Comment