REF definitions

51. For non-text, or practice-based outputs (including patents, software and standards documents) all sub-panels require the submission of a description of the research process and content, where this is not evident within the output (maximum 300 words).
52. For reviews, sub-panels welcome the identification of the original research or new insights reported, to assist with the assessment of research quality (maximum 300 words).
53. [Sub-panel 11 invite] factual information to be provided about the significance of an output that is not evident within the output itself (maximum 100 words). This could include, for example, additional evidence about how an output has gained recognition, led to further developments, or has been applied.
54. HEIs are instructed to ensure that such evidence is succinct, verifiable, and externally referenced where appropriate. Where claims are made relating to the industrial significance of the output, the name and contact details of a senior industrialist must be given to allow verification of claims. Information provided should not comprise a synopsis of the output or a volunteered opinion as to the quality of the output, and information provided that is of this nature will be disregarded. It is expected that in most cases, sufficient information will be provided in significantly fewer words than the 100 word limit.
55. Information provided must not include citation data: any panels that make use of citation data will be provided with the data by the REF team. Sub-panels will take no account of any citation data provided directly by the HEI. Information not relating to the significance of the output, for example co-author contribution (other than as requested in paragraph 44), should not be included, and sub-panels will take no account of any such information submitted.

Some examples of additional information submitted to the 2008 RAE. Note that then it was acceptable to summarise the paper and provide citations, which you are no longer asked to do.

• Steve Furber (2014 panel chair)
• Manchester
• Plymouth
• Southampton
• York

78. Category A staff are defined as academic staff with a contract of employment of 0.2 FTE or greater and on the payroll of the submitting HEI on the census date (31 October 2013), and whose primary employment function is to undertake either ‘research only’ or ‘teaching and research’.

82. Category C staff are defined as individuals employed by an organisation other than an HEI, whose contract or job role (as documented by their employer) includes the undertaking of research, and whose research is primarily focused in the submitting unit on the census date (31 October 2013).

The old "Category B" (Category A staff who left the university before the census date) and "Category D" (Category C staff who have left) are no longer considered by the REF.

Full details in REF 02.2011

60. Sub-panels 7, 8, 9 and 11 will make use of citation data, where it is available, as part of the indication of academic significance to inform their assessment of output quality.
61. Where available, the REF team will provide citation counts (at a pre-determined date) for research outputs submitted in the UOAs identified in paragraph 60. These sub-panels will also receive discipline-specific contextual information about citation rates for each year of the assessment period to inform, if appropriate, the interpretation of citation data.
• Note: REF have signed a deal to use the Scopus database for this purpose. However, Portsmouth does not yet have access to this.The ISI Web of Knowledge (which we do have access to) is broadly equivalent.
62. In addition to the citation data provided by the REF team, Sub-panel 11 only intends to make use of Google Scholar as a further source of citation information.
63. For the sub-panels identified in paragraph 60, citation data will inform the assessment as follows:
1. Where available, citation data will form part of the process of assessment of academic significance. It will be used as one element to inform peer-review judgements made about academic significance and will not be used as a primary tool in the assessment.
2. The absence of citation data for an output will not be taken to mean an absence of academic significance.
3. Sub-panels will be mindful that for some forms of output (for example relating to applied research) and for recent outputs citation data may be an unreliable indicator. Sub-panels will take due regard of the potential equalities implications of using citation data.
4. Except for reference to Google Scholar by Sub-panel 11 (as set out at paragraph 62), the sub-panels will use only the citation data provided by the REF team and will not refer to any additional sources of bibliometric analysis including journal impact factors.

41. All sub-panels expect that an author who has claimed co-authorship of an output will have made a substantial contribution to it, and may request audit information to substantiate this. Neither the order of authorship nor the number of authors will be considered important.
42. Sub-panels do not require the submission of textual information about individual co-authors’ contributions to co-authored outputs. These sub-panels will not take account of the individual author’s contribution to an output in assessing the output, judging each output on its merits independent of authorship arrangements.
43. A co-authored output may not be listed against more than one member of staff returned within the same submission, irrespective of the number of submitted staff in that submission who are co-authors of the output. A co-authored output may be listed in different submissions, either from the same or from different HEIs.

85. Early career researchers are defined as members of staff who meet the criteria to be selected as Category A or Category C staff on the census date, and who started their careers as independent researchers on or after 1 August 2009. For the purposes of the REF, an individual is deemed to have started their career as an independent researcher from the point at which :
1. They held a contract of employment of 0.2 FTE or greater, which included a primary employment function of undertaking ‘research’ or ‘teaching and research’, with any HEI or other organisation, whether in the UK or overseas, and
2. They undertook independent research, leading or acting as principal investigator or equivalent on a research grant or significant piece of research work. (A member of staff is not deemed to have undertaken independent research purely on the basis that they are named on one or more research outputs.)
86. The following do not meet the definition of an ECR (this list is not exhaustive):
    1. Staff who first acted as an independent researcher while at a previous employer – whether another HEI, business or other organisation in the UK or elsewhere – before 1 August 2009, with a contract of 0.2 FTE or greater.
    2. Staff who first acted as an independent researcher before 1 August 2009 and have since had a career outside of research or an extended break from their research career, before returning to research work. Such staff may reduce the number of outputs submitted according to paragraph 92a.iv. (career breaks).
    3. Research assistants who are ineligible to be returned to the REF, as defined in paragraphs 80-81.
87. ECRs may be submitted with fewer than four outputs without penalty in the assessment, as described in paragraphs 90-100 and in the panel criteria and working methods documents. Regardless of whether or not they are submitted with fewer than four outputs, all staff included in a submission who meet the definition of an ECR must be identified as ECRs in the submission. This is to enable the funding bodies to analyse the selection rates for ECRs across the sector as a whole, as part of our wider analysis of selection rates. To enable this analysis, the HESA staff return for 2013-14 will include a field for HEIs to identify all academic staff on ‘research’ or ‘teaching and research’ contracts who meet the REF definition of an ECR.

REF allows staff to submit fewer than 4 research outputs (without penalty) "where their circumstances have significantly constrained their ability to produce four outputs or to work productively throughout the assessment period".

This might include:

1. Clearly defined circumstances, which are:
1. Qualifying as an ECR (as defined at paragraphs 85-86).
2. Part-time working.
3. Maternity, paternity or adoption leave. (Note that maternity leave may involve related constraints on an individual’s ability to conduct research in addition to the defined period of maternity leave itself. These cases can be returned as ‘complex’ as described at sub-paragraph b below, so that the full range of circumstances can be taken into account in making a judgement about the appropriate number of outputs that may be reduced without penalty).
4. Secondments or career breaks outside of the higher education sector, and in which the individual did not undertake academic research.
2. Circumstances that are more complex and require a judgement about the appropriate number of outputs that can be reduced without penalty. These circumstances are:
   1. Disability. This is defined in Part 4, Table 2 under ‘Disability’.
   2. Ill health or injury.
   3. Mental health conditions.
   4. Constraints related to pregnancy or maternity, in addition to a clearly defined period of maternity leave. (These may include but are not limited to: medical issues associated with pregnancy or maternity; health and safety restrictions in laboratory or field work during pregnancy or breastfeeding; constraints on the ability to travel to undertake fieldwork due to pregnancy or breast-feeding.)
   5. Childcare or other caring responsibilities.
   6. Gender reassignment.
   7. Other circumstances relating to the protected characteristics listed at paragraph 190 [reference to the Equality Act 2010].

In the context of the research environment, sustainability will be understood as consideration of:

• leadership,
• vision for the future and
• investment in people and infrastructure and,
• where appropriate for the subject area, the extent to which activity is supported by a portfolio of research funding.

37. All forms of research output will be considered equitably in terms of the assessment, with no distinction being made between the types of output submitted, nor whether the output has been made available electronically or in a physical form.
38. The main panel welcomes all forms of output submitted to its sub-panels, including:
    • books, book chapters and research monographs
    • conference contributions and reports
    • new materials, devices, products and processes
    • patents
    • published papers in peer-reviewed journals
    • software, computer code and algorithms
    • standards documents
    • technical reports, including confidential reports.
39. These are provided as examples of outputs that might be specifically relevant to Main Panel B but should not be regarded as an exhaustive list.
40. In relation to all forms of output, submitting HEIs should be mindful that the purpose of the assessment of research outputs is to assess the quality of original research reported. In particular, sub-panels will accept the submission of review articles only where they contain a significant component of unpublished research or new insight. Such outputs will be judged only on original research or new insights reported.

There are 4 main panels sub-divided into 36 sub-panels covering the range of academic disciplines.

Computer Science and Informatics is Unit of Assessment 11 - part of Main Panel B.

Sub-panel 11 defines its scope as:

12. The UOA includes the study of methods for acquiring, storing, processing, communicating and reasoning about information, and interactivity in natural and artificial systems, through the implementation, organisation and use of computer hardware, software and other resources. The subjects are characterised by the rigorous application of analysis, experimentation and design.
13. The sub-panel expects submissions in this UOA from all areas of computer science and informatics, as defined above, and expects that the majority of the research activity submitted will have made a direct contribution to the UOA as characterised in the UOA descriptor. It recognises and welcomes, however, the increasingly interdisciplinary nature of research in this area, and expects that submissions may contain outputs that make contributions to computer science, informatics, and other disciplines.

• provides an encouraging environment for research,
• has an effective strategy,
• is engaged with the national and international research and user communities, and
• is able to attract excellent postgraduate and postdoctoral researchers.