Self evaluation

ORPHEUS Evaluation Certificate – a tool to facilitate self-reflection of doctoral training programmes

BACKGROUND
The ORPHEUS “Standards for PhD education in biomedicine and health sciences in Europe” was published together with AMSE  and WFME  in 2012. The document expressed a consensus, developed over many years, between around 100 biomedical and health science institutions all across Europe concerning the aims for the outcomes and content of PhD programmes. It was envisioned that the document would be used by institutions to determine to what extent their PhD programmes were consistent with this consensus, and to implicitly indicate how their PhD programmes might be improved. This document has now formed the basis for a new document “Best Practices in PhD Training”. These documents are thus tools that can be employed for self-reflection by institutions.

PURPOSE
It is now the intention of ORPHEUS to provide institutions with a comprehensive and flexible procedure to assist this process of self-reflection. This procedure is based on an ‘ORPHEUS Evaluation Certificate’ and the procedure has various stages. It will be the decision of the individual institution as to what stages they wish to use, or alternatively which stages are possible to complete in light of national rules. The procedure is organized by the ORPHEUS Labelling Board that consists of persons who have practical experience as heads of graduate schools. Whether all stages of the procedure are conducted or not, the consultative aspects of the evaluation makes the process advantageous.
THE EVALUATION PROCEDURE
The suggested procedure is as follows:

STAGE 1
Completion of an ORPHEUS self-evaluation questionnaire and submission of the completed questionnaire to ORPHEUS. It should be clarified which stakeholders have been involved in completing the questionnaire.

STAGE 2
Evaluation of the questionnaire by two members of the ORPHEUS Labelling Board, or other persons appointed by the Board, who will provide the institution with feedback about compliance/non-compliance with the ORPHEUS standards and suggestions about how the programme could be improved. These points will appear on the ORPHEUS Evaluation Certificate. This service will be provided without fee.

STAGE 3
Given that a written self-evaluation is unlikely to provide a full picture of the institution’s PhD programme, ORPHEUS can organize a 1½-day site visit to facilitate the self-reflection process. The facilitators will be members of the Labelling Board or other persons with practical experience as heads of graduate schools appointed by the Board. During the site visit the facilitators meet with relevant stakeholders, e.g. head of graduate school, dean, head of graduate school administration, programme directors, supervisors, PhD candidates etc. The institution will be asked to cover the travel and local costs of the facilitators. There is no fee for the facilitators. The institution can choose whether to have one or two facilitators, and whether they would also like the team to include a PhD candidate (chosen from the PhD candidates associated with the ORPHEUS Executive Committee).

As for stage 2, ORPHEUS will provide the institution with an ORPHEUS Evaluation Certificate with a list of recommendations regarding the extent to which the institution complies with the ORPHEUS standards and suggestions for the institution to consider regarding measures to possibly improve the programme.

OUTCOMES
Every institution will receive an ORPHEUS Evaluation certificate to recognise that the process has been conducted.
For institutions that on the basis of stage 3 are determined to fully comply with the Basic ORPHEUS standards, the institution may if it wishes be awarded an ORPHEUS label diploma. There is no fee.
The procedure may be iterative. For example, if an institution does not fully comply with the standards under stage 3, then it may make the requisite changes and re-apply for consideration under stage 2 or 3.