The University of Sheffield: PowerPoint template

The Development of REF:Current Thinking and ProgressGeof TomlinsonPro-Vice-Chancellor (Research)University of Sheffield

The end of RAE2008: lessonslearnt• There were 6 sub-panels under Panel G(Engineering)• Approximately 25% of the overall qualityprofile was awarded on the basis ofmetrics• At the last main Panel G meeting severalaspects were identified5/15/2009 © The University of Sheffield

o Confidence was expressed in assessment ofthe research outputs including appliedresearcho Converting the two metrics of PhDs awardedand research income into a profile wasdifficult5/15/2009 © The University of Sheffield

o Small submissions skewed the metricsapproach: size of return vs quality isimportanto Discrimination arising from profiling acrossgroupso Early career researchers – issues with theassessmento Rounding to 5% in quality levels5/15/2009 © The University of Sheffield

REF: Expert Advisory Groups• Drawn largely from Chairs and members of RAEpanels plus nominations from RCs and other keystakeholders• Over 100 members with broad coverage (5 fromEngineering)• 3 rounds of meetings covering all key features ofREF:February 2009April/May 2009June 20095/15/2009 © The University of Sheffield

Proposed timetable:• Up to summer 2009:- conclude the bibliometrics pilot- take advice from EAGs- hold discussions with a wide range ofstakeholders• Autumn 2009:- publish proposals on the key features ofthe REF for consultation with the sector5/15/2009 © The University of Sheffield

Proposed timetable (contd):• Early 2010:- announce outcomes of the consultation• 2013:- complete full assessment of all subjectsthrough the REF to drive funding from20145/15/2009 © The University of Sheffield

Questions being considered bythe EAGs• Panel/subject configuration options• How can HEIs reflect the impact ofresearch• Key generic elements of assessment• Types of evidence and indicators• Options for combining evidence/indicatorsto produce overall quality profiles5/15/2009 © The University of Sheffield

Questions being considered bythe EAGs (contd)• Eligibility• Output creditation/number of outputs• Number of expert panels• Consistency aspects across panels• Scope for reducing burden of data collection• Alignment of REF to an institution’s researchmanagement5/15/2009 © The University of Sheffield

Some reflections from the EAGs• Scope for fewer panels but:Increase in members per panel, not necessarilyreduce burdenCoverage/lack of coherenceLarger disciplines in broad panels, impact on‘smaller’ disciplinesLimited scope for fewer, broader panels5/15/2009 © The University of Sheffield

• Impact:Important to take more explicit accountHEIs might produce a portfolio involving longerand shorter term impacts using case studiesEnabling impact: KT engagement with usersand the publicEligibility:Retain this but consider the use of a moreconsistent methodology5/15/2009 © The University of Sheffield

• Outputs:4 outputs suggested to be roughly right but someflexibility needed, e.g. ‘2 in A&H but 4 in STEM’• Output creditation:To follow the individual• Consistency:Use same % outputs for all panelsPropose the use of a ‘template’ for RA5 and betterguidelinesInternational experts on all panels5/15/2009 © The University of Sheffield

• Burden reduction:o Communication needed ASAP on what datacollection to enable HEIs to introduceappropriate IT systemso High level aggregation will reduce worko Use of more pro-formas5/15/2009 © The University of Sheffield

ToDiscoverAndUnderstand.