1 Introduction: A Spectrum of Feedback Choices

A good educator knows when it is time to speak, and when it is time to listen. As we recognise that education is much more than just transmission of data, we recognise that educators have a responsibility to design a system of learning which includes regular feedback and evaluation systems which improve that design. Through feedback we seeking meaningful data of student learning on which to form judgements and through evaluation we plan and make strategic decisions on the basis of that data. These are fundamentally questions of educational design not administrative policy.

If there is one core theme of this manual it is the importance of moving beyond the strictures of compliance, audits and institutional review, to instead look at evolving good practice by which an educator is tapped into the learning environment, makes connections with learners and shapes their practice accordingly. Academics are all skilled researchers and it is through our feedback and evaluation procedures that we turn that critical lens on our teaching practice to best enable learning.

Innovation and maintenance of excellence is built on a foundation of feedback which is effective, honest and based on good evidence. When we embrace feedback from a variety of different points of view we avoid complacency, we continually challenge ourselves to come up with better solutions to problems. As educators we have available to us a whole spectrum of feedback choices, but unfortunately we have come to rely heavily on one, the anonymous student survey.

What this manual is not.

This is not a guide to providing effective feedback to students on their progress or their assessment items. It is, instead, an overview of the strategies by which a course (or a unit of study) is evaluated and the feedback mechanisms which provide the evidence to make these decisions. It is unfortunate that terms such as ‘feedback' or ’evaluation' can have confusingly broad meanings. Nevertheless, there remains some overlap between the feedback educators give to students and that which is sought from them as these form different aspects of the communication relationship which is explored in section 6 of this manual which looks at the feedback possibilities of the assessment process.

Moving Beyond the Student Survey

It is unfortunate that the anonymous student survey has such a dominant position on the feedback and evaluation landscape. There is nothing wrong with the student survey in itself and it will be discussed in section 5, but it is only one technique among many and a singular focus risks missing much of the texture and flavour of the big picture. These surveys are often used for procedures that they are not designed for, such as evaluating teacher performance or deciding on course content. This manual aims to provide a broad menu of choice along with practical tools for implementing feedback, reflecting on it and closing the loop by acting appropriately.

The flaws of the anonymous student survey will be discussed in section 5 and one particular issue is that it creates a deceptively simple one-line figure, an apparent metric that that can be easily tied to quality policies and procedures but this can a crude, often erroneous use of data where it is used to answer questions that the survey method does not ask. Further, this can entrench a set of adversarial relationships, between academics, students and administrators where more diverse feedback measures may build collaborative bridges instead. Badly implemented or designed feedback mechanisms can demoralise staff and place evaluation as a matter for external oversight and compliance. Evaluation is, rather, a matter of good practice which is owned by the learning community of both educators and students. If seen as extra work, outside of everyday practice, they become a chore.

When considering feedback, like weighing research sources more generally, it is always sound practice to seek a diversity of points of view and triangulating data to confirm reliability. This allows us to not only embrace different perspectives but also different methodologies, from the strictly empirical through to experiential practice-based knowledge in order to build a textured, nuanced model backed up by evidence.

Feedback is not a process external to good teaching. All educators use a variety of informal feedback measures in the class-room, asking questions, looking around for nods of recognition, talking to students about their understanding. An effective feedback method builds on this relationship and seeks an efficient process by which information can be obtained, recorded, tested and reported efficiently. In other words, it is about coalescing features of the informal teaching and learning relationship into sound data as the basis of good decision making as well as credible research.

Feedback Method Capsules

Throughout this manual we will be highlighting different feedback methods and to provide a quick summary of their uses and weaknesses. The following format will be used:

Title:

Self-explanatory

In a nutshell:

2 or 3 sentences explaining the method and how it is used

Example Questions:

What kinds of feedback questions you might ask using this method?

Reporting:

How do you communicate your evaluation of the feedback back to the stakeholders? How do you close the loop?

Pros:

The best features

Cons:

Potential problems

Caution:

Not cons as such, but things to look out for, such as difficulties in gaining reliable data, or ways in which reporting can be skewed.

The Benefits of Having Good Feedback and Evaluation Systems

Honest, evidence-based feedback can be used for a variety of purposes, beyond course review and reporting obligations:

• Feedback forms the starting point of reflective self-evaluation, a capacity which is developed through practice and is valuable in understanding educational innovation.
• Keeping a record of feedback forms a historical record that can inform others and act as a memory prompt for the individual, particularly when building on innovation that may have be implemented a year earlier.
• A good feedback and evaluation process allows you to share information with colleagues across the course and in the academic community generally. Across a semester it is useful to tap into comparative feedback from the students in different learning contexts and from different year levels.
• Records of feedback and evaluation form the core of valuable research data. If this is collected through regular embedded procedures it is easier than having to seek out data for specific research projects.
• Where diverse feedback measures are used, these can provide context and texture for more formal metrics, to explain and understand the data.
• An effective feedback culture enhances communication and can have benefits in connection to other policies such as determining students at risk or implementing equity policies
• Being able to collect, reflect on and analyse feedback data creates a solid foundation for professional development, awards and promotion, a more complete picture of the educational professional than simple metric data can provide.

Closing the Loop: Embedding Feedback and Evaluation into Practice

One of the reasons that evaluation is a dreaded process is that it typically involves the imposition of an obligation, from somewhere external to the classroom and on top of the existing relationship that has been built there. But this is not necessarily the case – feedback and evaluation systems can come from the grassroots and be embraced with enthusiasm by teachers and learners, especially when it has a clear and direct impact on everyday activities. Further, developing a strong culture of feedback that is listened to, respected and acted on can make the more formal types of feedback (such as anonymous surveys) less adversarial, more relevant to students and more constructive.

We must make time and space for feedback. As such it is useful to get in the habit of regularly seeking feedback as a matter of habit. Different types of feedback can be used to keep things fresh from big questions about delivery methods to simple choices around the topic to be explored in the next section. Students will respond more enthusiastically if they are expecting to give feedback and know that it will be listened to and acted on. Some low key easy to implement strategies are considered in section 7.

For some teachers, the idea of changing practice on the basis of feedback may seem troubling, particularly where this is connected to the problematic ’customer satisfaction' mode of feedback. Educational design is not a majoritarian process, which is why feedback needs to be carefully designed around issues where flexibility would enhance learning, guided by academic judgement. Do not ask open ended questions where feedback is not going to have the potential of influencing the outcome, where other factors such as course accreditation have the final say. We need to commit to the process if we want respondents to do so as well.

Acting on feedback is sometimes called ’closing the loop' but these closed loops also form the foundation for the next cycle of feedback. It is important that feedback and evaluation processes are open and reported back in a timely way and that feedback is acted on in a way that shows good faith in the process. This does not of course mean simply doing what the feedback demands, academics must exercise judgment in weighing options and alternatives, and then communicate the reasons for decisions in a way that respects feedback even if a different choice is made.

Ways of Closing the Loop
Build it into your regular review cycle. When preparing an annual report, make sure you have expressly referred to feedback and described the ways in which you are responding to it.

Provide feedback on assessment tasks, not just to individual students but provide global feedback about what your expectations were, how the assessment submitted measures up and what you (and the students) will be doing in response.

When feedback occurs in cycles, be sure to let the next cycle of students know what feedback you have received on previous rounds and how you have responded. This could be a paragraph in your unit outline where you describe previous feedback and its result. This can stop students returning to issues that have been settled in the past or are the topic of current innovation, fostering an environment where feedback is heard and acted upon.

When you get data, such as from learning analytics, make sure you share it and provide opportunities for others to comment, even if it is just informal.

Keep the loop going, feedback circulating, use a juncture at which you give feedback to the students as an opportunity to seek further feedback from them.

The New Evaluation Landscape

There is a new, less-adversarial, evaluation landscape emerging from changes to universities and innovations in the way that we gather and share information. The predominant focus on anonymous student surveys has been troublesome for a number of reasons and contemporary evaluators seek feedback from a spectrum of different sources and methods relevant to a broader range learning outcomes beyond the mere transmission of information.

There have been several game changers that have pre-empted this evolution:

• The orientation to ‘whole of course’ evaluation has meant that we are no longer focussed on the individual unit of study as the primary nexus of learning. After all students experience their educations holistically, not as a number of disconnected experiences.
• Rejection of the neo-liberal idea of education as a consumer commodity has meant that the customer service flavour of many anonymous student surveys has had to be revisited and updated for a re-orientation toward student needs
• Students contend with regular survey requests at university, from telemarketers and in other aspects of their life, they suffer from survey fatigue.
• Evolving information technology makes it much easier to share information, to quickly gather and collate feedback and to provide updates on developments.
• Students and educators who have grown up with interactive media, from online services through to games, have an expectation that learning is a shared and interactive process and that this interaction should allow them to explore and customise their own personal learning environment.
• The increasing recognition of ethical, reflective practice means that professional educators should be judged on their ability to critically reflect, to solve problems and to make judgements rather than to seek to measure performance with a crude measuring tool.

What is feedback?

The difference between dynamic and static systems is that a dynamic system evolves on the basis of feedback received. Feedback occurs in cycles, where each stage of innovation is built on the foundation of previous cycles. In this manual we take a wide definition of feedback which includes any information we seek, collect and record, including both qualitative and quantitative data that forms the evidence basis on which we make decisions about learning design. Evaluation on the other hand refers to the systems by which we process and analyse feedback.

The anonymous student survey still has an important role in measuring aspects of the student experience and to identify hot spots, but it has been ingrained in a position where it is used for more purposes than its methodology can support. In particular, the widespread use of student opinion experience in a punitive fashion, to performance manage staff and to deny promotion opportunities has been criticised as a misuse of data.¹

By analogy, if a medical clinic used the client suggestion box as the sole means to judge the effectiveness of the practice and staff were promoted on the basis of this information alone, we can see the impact this would have of the ability to make professional judgments. A health professional's service to clients is very important but those clients may be in no position to judge many of the professional aspects of the job. It would be negligent for a manager to rely completely on the suggestion box when evaluation effectiveness and universities risk making the same mistake of rule by the tool.

Resourcing Evaluation

Many universities are not very good at seeking and evaluating feedback so it can feel a bit daunting to introduce new systems.

Don't try to do everything at once, make sure you stage your feedback plan over a three or five year cycle rather than every year.

Be active in changing your departmental culture to see feedback as a part of teaching rather than as an external pressure on it.

Redirect resources to feedback from areas where you know are not making an impact. If you are recording lectures that students do not access, think about ways of building interactive activities that you will also gather feedback data from.

Crowd source and use networks where you can, work with colleagues, activate the students to be co-producers of their educational experience and therefore active in the feedback and evaluation process.

Evaluate early rather than late. Where there is a danger that feedback may accelerate into complaints these will drain significantly more resources.

This new landscape calls for a rethinking of feedback and evaluation systems, bearing in mind social and technological changes. Education is a collaboratively constructed experience and all of the stakeholders (academics, students, community and industry) need a voice in the process, to be sought at the most relevant juncture and least intrusive way.

Evaluation is no longer merely a matter of compliance, of the end of semester survey imposed from above and reluctantly accepted with resignation. Seeking feedback, evaluating, acting on it and communicating this back to stakeholders is now core business for educators in a way that supports them in their work, not a distraction from it. Evaluation is not about allocating blame on individuals, but about a course-level team proactively and honestly identifying issues and seeking collaborative answers to problems.

Key Points

• Methods of obtaining feedback and the creation of an evaluation process are research design choices not administrative procedures
• Effective feedback and evaluation is built on a spectrum of different methods and data sets in order to build a more complete picture.
• Feedback provides multiple benefits and is a practice best embedded in everyday practice in order to build a strong relationship of respect.
• When we seek feedback we must ’close the loop' by evaluating the evidence, making an informed decision and communicating the decision and reasoning.

1. For an exploration of this see Blackmore, Jill (2009)’Academic pedagogies, quality logics and performative universities: evaluating teaching and what students want', Studies in Higher Education, 34(8), 857-872