Asking if rejection feedback is helpful

Illustration with the text 'Is this feedback helpful?' and two buttons labelled 'Yes' and 'No'

When candidates receive a rejection from a provider along with some feedback, we are now asking candidates whether this feedback is helpful or not.

This is an experiment, to see whether we get sufficient responses to help us make decisions about ways to try and improve the helpfulness of this feedback.

We started asking this question on 15 November 2022.

The issue

Despite changes to how we ask providers to give feedback to candidate when rejecting their application, reviewing the actual feedback given shows a lot of short, unhelpful answers.

Previous research with candidates has revealed that having an application rejected can make candidates feel dejected, and poor feedback can put them off from re-applying.

Deciding whether feedback is helpful or not is subjective, and so it is hard to assess whether any changes we make have been successful or not.

What we did

We added a question on the service which appears below any rejection feedback given.

Screenshot with the heading 'Your applications' followed the text 'Your applications were unsuccessful. If now’s the right time for you, you can still apply for courses that start this academic year.' and a green button labelled 'Apply again'. Beneath this, two boxes are shown, each one has the title of a provider and a course and the status 'Unsuccessful'. The first one contains the feedback 'Subject knowledge: You did not demonstrate sufficient knowledge of the primary curriculum'. Beneath this is a line and then a question 'Is this feedback helpful?' followed by 2 grey buttons labelled 'Yes' and 'No'. The second box is similar, but with the feedback 'Peformance at interview: Poor answers to questions about motivation'.

There is no requirement for candidates to answer the question.

Once the question has been answered, the page is reloaded and the question is replaced with text to show how they answered.

Screenshot with the heading 'Your applications' and the same text as the previous screenshot - except that beneath the first set of feedback is the text 'You said that this feedback is helpful.' and beneath the second one 'You said that this feedback is not helpful.'

There is no confirmation step, and no way to change their answer, as we wanted to keep it as simple as possible.

Initial results

As of 28 November 2022, almost 2 weeks after having first asked this question, there have been 379 responses. Of these:

  • 191 (50%) said the feedback was helpful
  • 188 (50%) said the feedback was not helpful

The response rate so far has been around 12%, for rejections given since the feedback launched.

Most candidates who have rated their feedback have done so within a day of receiving it.

Further considerations

Once we have received more responses, we can start to look for patterns in the data by seeing if the helpfulness of rejection feedback varies by:

  • category of rejection received
  • type of training provider
  • individual training providers
  • whether or not the candidate had an interview
  • candidate characteristics

We can also then consider ways to improve the helpfulness of the feedback.