14 Evaluating Teaching

The primary purpose of this manual is understanding and enhancing learning by focusing on the student and their learning context. Teaching is a part of that context and this section examines some of the methods of evaluating teaching with an emphasis on the role of the teacher as an autonomous professional, rather than as a service provider.

Teaching quality, like learning, is often assessed through student surveys. The same cautions and caveats also apply here, complicated by the need to define what constitutes 'good teaching' and the often limited comparative experience of students. As with learning evaluation we recommend a spectrum of techniques that will ensure good quality feedback from a variety of different sources and perspectives.

The learning/teaching nexus

Contemporary universities are said to be student centred, with a focus on the learning needs of the students rather than the teacher at the centre of the system. Traditionally we assumed that good quality teaching would automatically enhanced learning and conversely that poor teaching would be an impediment. As critical learning researchers we should be prepared to test these assumptions and acknowledge that students all learn differently and respond differently to various teaching styles.

While quality of teaching may not have the direct impact on student learning that we would like to assume, it is safe to say that good quality teaching enhances the satisfaction of the student experience, improves engagement and raises morale for the students and the teachers alike. Improved teaching may not be the magic bullet to fix student learning problems, but it will create a better learning environment overall.

The anonymous student survey is one of the more contentious forms of student feedback and some teachers are concerned that this amounts to little more than a popularity contest and that students find it difficult to distinguish the teaching skill of the individual teacher from the course and unit context in which they operate. As with course-based student surveys the design of the instrument is crucial in asking students questions that they are able to answer from their perspective, to focus on their experience rather than overall judgements of good or bad.

It is also crucial to maintain focus on the teacher as an autonomous professional, in designing teaching evaluation procedures that respect professional choices and allow for reflection and autonomous improvement, rather than focus on perfomativity. Poorly designed feedback amounts to little more than a 'customer satisfaction' box and fail to focus on real issues of improvement and ongoing professional development.

What is good teaching?

This is a dense and complex issue, with a diversity of perspectives from different disciplines. Most universities attempt to separate teaching quality from course quality questions but in practice this division can be arbitrary and difficult to maintain, especially in single teacher units. While there is no single gold standard, it is possible to identify different themes:

1. Presentation and performance.

Is the teacher an effective speaker, who engages their audience and delivers learning material in a way that is clear and comprehensible? Does the teacher motivate and enthuse the audience, are they able to both tell a good story and clearly explain complex theoretical material?

2. Design of learning materials

Is the teacher able to organise and present learning materials in a way that is clear, comprehensible and engaging? Does the structure of learning materials enhance learning? Can the lecturer design the display of information in an accessible way including visualisation techniques where relevant?

3. Leadership and group skills

Is the teacher able to manage the learning community, to facilitate learning and discussion, to coordinate individuals and groups? Is the teacher able to balance the competing interests of the institution, the learners and academic staff? Do the needs of the many balance with the needs of the one? Can the teacher set clear and unambiguous policy but also apply it in a way that is fair and respects individual circumstances?

4. Academic skills

Does the teacher demonstrate command of their subject material in a way that is accessible to students? Does the teacher balance cutting-edge research based learning with the needs of learners of learners to build basic skills and knowledge frameworks?

5. Pastoral care and interpersonal management skills

As one of the important interfaces between the institution and individual learners, is the teacher able to understand individual issues and solve problems? Is the teacher empathic, do they demonstrate emotional intelligence? Are they able to be firm or flexible where necessary?

6. Interactive design skills

This is an emerging skill set as learning environments become more student directed, often mediated by communications technologies. Can the lecturer design and manage learning activities that are engaging and allow different paths of exploration through the materials? Are students free to make legitimate choices within the system but also supported where necessary so that they do not feel lost? A contemporary learning experience should feel less like a roller-coaster, stuck on rails and more like a wilderness walk where trails and park rangers help you make choices and provide a safe environment.

Timing effective teaching evaluation

A spectrum of teaching evaluation techniques will provide a rich tapestry of feedback data and the proper management of this data is critical if teachers are to feel valued and also be open to improvement strategies. A top-down hierarchical approach is rarely effective and will not get the buy-in from teachers that is necessary for success. A peer-based approach that takes into account self reflection and professional networks will engage the teacher in their community of practice.

Timing is a crucial factor. Too often feedback is left until too late, where it is impossible to make changes and where relationships may already be damaged, particularly where inexperienced teachers are involved. A key problem with the end of semester student survey is that it occurs at the wrong time to make any effective changes for that group of students and when students have been galvanised into making a complaints it is more difficult to repair the relationship.

For these reasons it is important to be proactive, to provide guidance and support for inexperienced teachers (and others as well) so that issues can be discussed before they become sore points of contention. A respectful relationship is central to this process, no one likes “being managed” and evaluation must be designed to augment autonomous reflection, not to impose external judgement. Managers are frequently frustrated by managing negative feedback where teachers can be highly defensive and this is caused by a reactive rather than proactive evaluation design.

Methods

• Student surveys on teaching
• Teaching skill mentors & peer review
• Self assessment and teaching portfolios
• Recording and review
• Teaching skills professional development

Key points:

• Teaching evaluation should take place in a respectful environment with a focus on autonomous professionalism, reflection and self improvement
• Standards of “good teaching” draw on shared values, negotiated by context and embrace a variety of different factors and skillsets.
• It is essential that evaluation is timely and that it is oriented toward improvement rather than judgement.

Student surveys on teaching
In a nutshell:

Anonymous student surveys focussing on the individual teacher rather than course design.

Example Questions:

Does this lecturer present information in a way that I can readily understand?

Is spoken material delivered clearly and in a way that enhances comprehension?

Reporting:

Unlike unit surveys, these surveys are often confidential. Institutions differ in practice and some unions have concerns about student surveys being used for performance management.

Pros:

Anonymity may allow more honest feedback

Can challenge assumptions about what students want and what styles they best respond to

Cons:

Anonymity may be used as a shield for vindictive behaviour

Unless questions are framed carefully it may ask students for an opinion on matters that they are not able to answer

Caution:

This is a difficult technique to time effectively as feedback often occurs well after effective changes can be made.

It can be difficult to differentiate student feedback on the unit from that on the teacher.




























Self-assessment and Teaching Portfolios
In a nutshell:

A teaching portfolio collects evidence of teaching practices and the individual's reflection on that evidence. These portfolios may be a simple document/folio or may form part of an e-portfolio system.

Example Questions:

What areas of teaching am I most skilled at? What evidence do I have of this?

What are the gaps in my portfolio? What should I do to fill those gaps?

Reporting:

An individual can share as little or as much as they like from their portfolio.

Pros:

Effective for professional development and setting objectives

Extremely useful for awards and promotion as evidence is gathered over time and applicants do not have to start from scratch.

Cons:

Some teachers may find it difficult to make the time to work on their portfolio.

Caution:

There are definitely skills to be learnt in compiling and presenting a portfolio and some teachers may find the difficulty curve daunting and require further support.




































Teaching Skills Professional Development

In a nutshell:

Traditionally academics were hired on the basis of their knowledge and research rather than their teaching ability. Over time we have developed different ways of supporting teaching skill professional development, from formal teaching qualifications through to seminars and other support resources. Micro-credentialing and badging systems provide more immediate and gamified ways to evidence professional development activities and motivate further improvement. Further, most institutions run annual learning conferences which are another forum to engage in peer development networks.

Example Questions:

How do I learn to use social media in my teaching?

How do I demonstrate my skills in devising learning scenarios? How do I provide evidence of this?

Reporting:

Formal qualifications are added to the individual's CV and badging allows you to choose what to display in your e-Portfolio or CV.

Pros:

A comprehensive professional development framework both supports teachers and demonstrates that the issue is taken seriously by the institution.

Well structured professional development can allow teachers to set their own improvement objectives and plot a pathway.

Cons:

Professional development is expensive to run and it can be difficult to match activities to individual needs, particularly in a large organisation.

Caution:

Professional development needs to be strategic for the individual and the organisation. It can be difficult to reach the teachers who need it the most and development systems run the risk of preaching to the converted and 'PD junkies'.
















Teaching Skill Mentors & Peer Review

In a nutshell:

As with other areas of course design, having mentors specifically oriented toward teaching skills can enhance feedback. Mentors can observe teaching performance and review written materials and other resources to provide honest but sensitive feedback and advice. Mentoring may be individualised or it may form part of more extensive peer support networks.

Example Questions:

How effective are these changes I have made to the way I present these materials?

How should I deal with this problem which keeps arising in my classes?

Reporting:

All feedback is confidential.

Pros:

Personal and immediate support.

An effective way to build a stronger academic community and to share skills and experiences.

Cons:

Mentoring programs take time and resources to administer and if not taken seriously can further entrench feelings of isolation.

Caution:

Mentors are not always right, the relationship should be based on set principles of respect and mutual feedback. Bad mentors can impose power relationships that are not conducive to individual expression and development. For example, a cynical mentor can contaminate inexperienced teachers with their own negativity.

These processes work best if done with guidelines and parameters, many universities provide peer review and mentor templates to zero in on shared priorities and to provide structure.

Recording & Review In a nutshell: Where teaching involves spoken lectures or online delivery it can be useful to record proceedings and review them later, either by yourself or with a mentor/peer. Example Questions: How can I help students understand how these principles fit into the broader canvas? Do I really sound like that? Reporting: This technique needs to be confidential, even where recorded materials are already public. Pros: Can identify various aspects of performance that the individual is not aware of. Cons: Unless carefully guided it can be difficult to propose solutions to problems, particularly where they may arise from a lack of confidence. Can lead to a focus on form rather than content. Caution: Many people find this process to be very painful and are not used to responding to this sort of feedback. It can take time and support to develop the 'callouses' needed to examine personal performance in a positive and productive manner.