4.2.2: Guidance on the conduct of the annual review of taught programmes
Please note: this a record of the historic annual review process which Durham operated 2011/12-2016/17. From 2017-18, a revised, centrally-led process of annual monitoring will operate, as proposed within the RADAA report considered by Senate in June 2017.
1. This guidance sets out the areas that departments should consider as part of their annual review. The University's expectation is that each department considers all relevant areas, and that the consideration of these areas is documented in the minutes of a departmental committee or meeting.
2. Under each of the headings are a series of prompts to support departments in considering these areas. These prompts are not intended to be exhaustive, but rather to identify the types of issues that are likely to be relevant to each of the areas. In addition to the key questions and issues set out below, the University's Principles for the Development of the Taught Curriculum should be used as a reference point as part of the annual review process. All taught programmes are expected to be aligned with these Principles by 2014, and annual review is the key mechanism for ensuring that this alignment takes place and is maintained. In carrying out this consideration, departments may find Section 4.5 of the Learning and Teaching Handbook useful in supplementing the prompts set out below.
Recruitment and admissions
3. Key questions and issues include:
a. Is the department meeting its admissions quotas (undergraduate) and targets (taught postgraduate)? (Separate targets are in place for home/EU and international recruitment, and both should be considered). Are there particular programmes that are not recruiting the number of students anticipated, and if so what action will be taken to address?
b. For programmes that have been approved and introduced in the last three years, are these now meeting the projected recruitment levels forecast in the Stage 1 business case proposal?
c. Do the admissions requirements for the programme (both academic and English language) remain appropriate?
d. What is the ratio of applications to entrants? Is it rising or falling?
e. Is the actual number of applications rising or falling?
i. How does this compare with the national picture for the subject?
ii. Given the level of applications, is the entry requirement appropriate? Are enough offers being firmly accepted to fill available places?
iii. If applications are falling, are there still enough high quality applications to issue all available offers? Could the programme be made more attractive to potential students? or could the department publicise it more effectively?
iv. Would help from the Student Recruitment and Admissions Office or Strategic Marketing be useful?
f. Is the proportion of applicants to entrants similar in respect of:
i. School type (state maintained or independent).
ii. Low Participation Neighbourhoods.
iii. Gender (e.g. is the percentage of female students admitted similar to the percentage who apply?).
iv. Students declaring a disability.
v. Students declaring an ethnic minority background.
vi. Mature students (aged 21 years of age and over).
vii. If the proportion of students in any category admitted is lower than the proportion who apply, what is the reason for this? Do they get offers but turn them down?
viii. If so what might be done about that (e.g. in the post-offer open day arrangements or in maintaining contact with students who have received an offer)?
ix. Might the basis for making admissions decisions be making it difficult for a particular group of students to demonstrate their potential?
x. Could this be addressed in some way? How can more students from any particular background be encouraged to apply - or to take up offers once they are made? Would help from the Admissions and Recruitment Section be useful?
4. All departments are supplied with department-specific admissions data as part of the annual review process. Departments who wish to compare their data to national admissions trends in their subject area can, for undergraduate admissions, find national data on the UCAS website at http://www.ucas.ac.uk/about_us/stat_services/stats_online/annual_datasets_to_download/. Comparative data on widening participation (again at undergraduate level) is available at http://www.hesa.ac.uk/index.php/content/category/2/32/141/ .
Curriculum review and development: learning, teaching, assessment & feedback
5. Key questions and issues include:
a. Are the aims and learning outcomes for programmes and modules up-to-date, current and valid? Are students achieving these learning outcomes?
b. Does the curriculum support students in achieving and demonstrating the programme learning outcomes? Are there opportunities for the department to develop and enhance the curriculum?
c. Do the programmes have effective approaches to linking research and education? Can these be enhanced?
d. Do the modes of learning, teaching and assessment used by the programme remain appropriate? Are there examples of particularly effective, or innovative, practice that can be identified? Is there scope to disseminate these more widely within the department, faculty or University?
e. Are effective policies in place to provide students with feedback on their assessed work? Is there any evidence that suggests that the department needs to revise or develop these policies? Is formative work playing an appropriate developmental role?
f. [2016-17 Annual Review+] what proportion of feedback was returned on time to students? What action was taken where feedback was not returned within the agreed schedule, and what action will be taken in future years - if necessary - to further improve the timeliness of feedback on assessed work?
g. Have new national reference points relevant to one or more of the programmes been issued (for example a QAA subject benchmark statement, or requirements of an external accrediting body)? If so, you will need to consider the relevant programme(s) against any such external reference point and where necessary and/or appropriate revise the programme outcomes, curriculum or learning, teaching and assessment strategy in the light of this.
h. Are the programme specifications for the programmes up-to-date and accurate? Do they reflect changes to programmes and/or modules agreed through the curriculum development process?
i. Do the results of module evaluation questionnaires highlight any issues of concern or areas for improvement that the department needs to take forward and address?
j. Do the learning, teaching and assessment methods on the programme help to provide an equal opportunity for all students to achieve and demonstrate the programme learning outcomes? Are there opportunities for the department to enhance the curriculum so that it is more inclusive in design? (The University’s guidance notes on inclusive curriculum design will provide a reference point to aid reflection on this issue)?
6. Please note that as part of your annual review you must review and as necessary update your programme specifications. It may be that your programme specifications do not need to be amended, but you should always review programme specifications to confirm whether this is the case. By the start of the 2016/17 academic year, all programme specifications should be accompanied by an up-to-date curriculum mapping document setting out how the learning outcomes are delivered across the programme. A template for this purpose and example curriculum mapping document are available as appendices to section 3 of the Learning and Teaching Handbook.
Student progression and achievement
7. Key issues and questions include:
a. What is the pattern of transfer and wastage from and between programmes in the department?
b. In which levels of study do transfers and withdrawals occur? To/from which programmes do students transfer?
c. Are there any common factors (such as the A-level grades or subjects) of those who transfer or withdraw?
d. Are the levels of transfer and or wastage a problem or is it a facility planned into the design of the curriculum? If it is a problem what can be done about it - e.g. in the admissions process, in advising students about module choice, in curriculum design?
e. What is the pattern of degree classes awarded? For undergraduate programmes, is the number of 2:1s and firsts appropriate given the entry qualifications of the students? For taught postgraduate students, is the number of distinctions and merits appropriate given the entry qualifications of the students?
f. What is the correlation between students' entry qualifications (grades and subjects) and their eventual achievement (for example do students with particular subjects [at A level or GCSE for undergraduate students; at first degree level for taught postgraduate students] do better or worse?
g. Do you mark high enough up the mark scale for really excellent work?
h. Does the curriculum design offer sufficient opportunity for students to 'play to their strengths' in terms of subject area?
i. Do the modes of assessment allow students to demonstrate their full potential?
j. Are there differences in progression and attainment between different demographic groups? If so, could this be addressed through student support or more inclusive curriculum design?
Employability, skills and graduate employability
8. Key questions and issues include:
a. Do programmes allow students opportunities to develop and demonstrate a wide range of graduate skills? If not, what could be done to increase these opportunities?
b. Are students obtaining 'proper' graduate jobs? If not do you know how long it takes them to find more appropriate employment?
c. What sort of employers recruit graduates from the department? What sort of jobs do students from your department want? Do the skills students acquire from your department's programmes equip them for the sort of careers they are seeking? If not can programmes provide more of the skills students might need, perhaps through choice of optional modules?
d. What proportion of the department's students is going on to further study after graduation? Are there further steps the department could take to encourage students to consider further study in higher education?
e. Where does the department stand in relation to the University's strategic objective that each subject area should be in the top 10 or top 10% nationally (whichever is the more challenging) for graduate employability? If you have not reached this target, what steps will be taken to address this?
f. What actions could be taken to improve the employability of the department's graduates?
g. Are there further opportunities for the department to engage with the Careers, Employment and Enterprise Centre in order to improve the department's first destination statistics?
9. Where a department offers placement learning opportunities (which includes student exchanges, years abroad, sandwich years in industry and shorter term placements such as Students into Schools modules and professional placements such as those in education and social work), these need to be considered as part of the annual review. This review should talk place in the context of feedback from students, and from placement hosts
10. Key issues/questions include:
a. The student learning experience.
b. Student support.
c. Facilitating return to study at Durham.
d. Student progression.
e. Assessment (modes, standards, criteria).
11. In respect of student exchange agreements (including Erasmus exchanges) departments key issues/questions also include:
a. Are there any issues for the department in relation to incoming students (for example in relation to academic and language proficiency)?
b. Is the balance between the number of incoming and outgoing exchange students appropriate (departments may wish to consider this over a two to three year period, rather than just the last academic year)?
Feedback from students and Student Engagement
12. This element of annual review should be conducted with reference to the University’s Principles of Student Engagement and the department’s policy on obtaining and using student feedback.
a. What have been the key issues arising from Module Evaluation Questionnaires? Have effective action plans been put in place to address these issues? Are there any broader issues arising from individual MEQs, that need to be considered by the department?
b. What have been the key issues arising from the National Student Survey? Have effective action plans been put in place to address these issues?
c. Have the results of MEQs and NSS been shared with the SSCC? Have all students been sent a summary of feedback setting out the key issues raise in internal and external (for example NSS) questionnaires, and the action taken in response?
d. Are response rates for MEQs sufficiently high to allow useful conclusions to be drawn from the results? (The University expects a response rate of at least 50% for questionnaires, and if this has not been achieved the department will need to put in place an action plan to address this).
e. Has the department's SSCC(s) operated in line with the requirements of the University policy on this issue (available in Section 5.2.4 of the Learning and Teaching Handbook)?
f. What have been the key issues arising from the SSCC(s) over the course of the academic year? Have these been addressed effectively, or are there outstanding issues that need to be considered? Has a short summary of key issues and action taken as a result of the SSCC been prepared for circulation?
g. Are the department’s (formal and informal) mechanisms (including, but not limited to, SSCCs) for student engagement effective in capturing and representing student views? Is there a need for alternative mechanisms? Does the department’s policy on obtaining and using student feedback require revision? This element of the review should be conducted with reference to the Principles for Student Engagement, and in particular those Principles that state that mechanisms for student engagement should:
i. help the department to enhance its educational provision;
ii. facilitate timely identification and acknowledgment of student queries and concerns (in addition to their views as to the strengths of the department’s provision);
iii. be inclusive, taking into account the diversity of the student body. For instance, are mechanisms effective in identifying issues that may be specific to only certain groups of students, such as disabled students, international students and part-time students?
In line with broader policy on annual review, departments should ensure that they directly involve student representatives in formulating a response to this aspect of the review.
13. Student questionnaires are a valuable source of information about the strengths and weaknesses of individual modules and there is good practice in many departments within the University in terms of how the information derived from questionnaires is analysed and presented to Boards of Studies to ensure that good practice is disseminated and appropriate action taken. Summaries of each key feature to be considered should be prepared by each module convenor, to include the following aspects:
a. the quantitative feedback (average scores for each question);
b. strengths identified;
c. weaknesses identified;
d. action proposed by the module leader;
e. action which might have to be taken at a more general level - for example if the feedback suggested that there were problems with the relationship of the module to the wider programme structure.
14. This will assist with the dissemination of good practice as well as with addressing problems. Some departments set the analysis of questionnaire feedback in the context of other information about the module, such as:
a. a summary of information about student achievement within the module;
b. e.g. average marks (for coursework, examination work and overall);
c. trends in marks over the last three years;
d. changes made in the previous year and the effect this has had on the reception of the module.
Again, the module leaders could be asked to include this in a brief summary sheet for each module.
Complaints and Appeals
15. Where the department has dealt with student complaints or appeals on a formal or informal basis, it should consider:
a. whether there have been any increase or decrease in complaints/appeals (and if so, the causes for change), and any specific or substantive issues arising from complaints and appeals;
b. whether any issues arising from complaints and appeals might be addressed via improvements to the guidance for staff and students on informal and formal complaints and appeal procedures.
16. Key issues/questions include:
a. Are there any issues relating to learning resources that have been identified through the course of the year, or as part of other review activities? How have these issues been addressed?
b. Are there any outstanding learning resources issues that need to be progressed with the faculty via the annual planning round?
External examiner reports
17. The University operates a separate process for considering issues raised in the reports of external examiners, and departmental responses to these. Consequently there is no need as part of annual review to report specifically on external examiner reports. However, departments should consider the following key issues/questions as part of the annual review:
a. Where the department has been asked to address issues raised by external examiners, how effective have the responses been? Is further action necessary?
18. Please note, that it is a University requirement that all external examiner reports, and departmental responses to these, are shared with the SSCC(s) (see Section 6.4.7 of the Learning and Teaching Handbook).
Diversity and equality
19. Consideration of diversity and equality issues is addressed via some specific questions above, but should form part of all areas of the annual review. Consideration of diversity and equality issues should form part of all the other areas of the annual review. The central question that a department should be considering when doing this, is whether there is any evidence that that policy, practice or provision is having, or may have, an adverse impact on any particular student group and if so what action should be taken to address this.